com.imsl.datamining.supportvectormachine

Class SupportVectorMachine

java.lang.Object

com.imsl.datamining.PredictiveModel

com.imsl.datamining.supportvectormachine.SupportVectorMachine

All Implemented Interfaces:

Serializable, Cloneable

Direct Known Subclasses:

SVClassification, SVOneClass, SVRegression

public abstract class SupportVectorMachine
extends PredictiveModel
implements Serializable, Cloneable

Abstract class for generating a support vector machine.

Support vector machines are machine learning algorithms for classification, regression, and the so-called one class problem. This package contains SVClassification, SVRegression, and SVOneClass for each of these problems, respectively. Each formulation uses a kernel based optimization to find support vectors that optimally separate clouds of high dimensional data for the purpose of making predictions. SupportVectorMachine and all of the classes in this package are based on the LIBSVM package of Chang, Chih-Chung; Lin, Chih-Jen (2011).

See Also:

Serialized Form

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	SupportVectorMachine.ReflectiveOperationException Class that wraps exceptions thrown by reflective operations in core reflection.

Nested classes/interfaces inherited from class com.imsl.datamining.PredictiveModel

PredictiveModel.CloneNotSupportedException, PredictiveModel.PredictiveModelException, PredictiveModel.StateChangeException, PredictiveModel.SumOfProbabilitiesNotOneException, PredictiveModel.VariableType

Constructor Summary

Constructors

Constructor and Description
SupportVectorMachine(double[][] xy, int responseColumnIndex, PredictiveModel.VariableType[] varType) Constructs a support vector machine for a single response variable and multiple predictor variables.
SupportVectorMachine(double[][] xy, int responseColumnIndex, PredictiveModel.VariableType[] varType, Kernel k) Constructs a support vector machine for a single response variable and multiple predictor variables.

Method Summary

Modifier and Type	Method and Description
void	fitModel() Fits the model to the training data, i.e, trains the support vector machine.
double	getConvergenceTolerance() Returns the convergence tolerance.
protected Kernel	getKernel() Returns the kernel object being used in the optimization.
double[]	getKernelParameters() Returns the kernel parameters.
protected SVModel	getModel() Returns the model object.
double	getNuParameter() Returns the value of the \(\nu\) parameter.
double	getRegularizationParameter() Returns the value of the regularization parameter, C.
double	getWorkingArraySize() Returns the setting for the work array size.
boolean	isNuFormulation() Returns the boolean to perform the \(\nu\)-formulation of the optimization problem.
boolean	isProbability() Returns the boolean to calculate probability estimates.
boolean	isShrinking() Returns the boolean to perform shrinking during optimization.
protected abstract SVModel	optimize(DataNode[][] x, double[] y, double[] w, int len, Kernel kernel) Abstract method to perform the support vector machine optimization.
double[]	predict() Returns the predicted values on the training data, i.e., returns the fitted values.
double[]	predict(double[][] testData) Returns the predicted values on the input test data.
protected abstract double[]	predictValues(SVModel model, double[][] attributeData) Abstract method for generating the predicted values using the fitted support vector machine model.
protected void	setConfiguration(PredictiveModel pm) Sets the configuration of PredictiveModel to that of the input model.
void	setConvergenceTolerance(double eps) Sets the convergence tolerance.
void	setKernel(Kernel kernel) Sets the kernel to be used in the optimization.
void	setKernelParameters(double[] kParams) Sets the kernel parameters.
void	setNuFormulation(boolean nuFormulation) Sets the boolean to perform the \(\nu\)-formulation of the optimization problem.
void	setNuParameter(double nu) Sets the value of \(\nu\) in the \(\nu\)-formulation of the optimization problem.
void	setProbability(boolean probability) Sets the boolean to calculate probability estimates.
void	setRegularizationParameter(double C) Sets the regularization parameter, C.
void	setShrinking(boolean shrinking) Sets the boolean to perform shrinking during optimization.
void	setWorkArraySize(double workSize) Sets the work array size.

Methods inherited from class com.imsl.datamining.PredictiveModel

clone, getClassCounts, getClassErrors, getClassErrors, getClassLabels, getClassProbabilities, getCostMatrix, getMaxNumberOfCategories, getMaxNumberOfIterations, getNumberOfClasses, getNumberOfColumns, getNumberOfMissing, getNumberOfPredictors, getNumberOfRows, getNumberOfUniquePredictorValues, getPredictorIndexes, getPredictorTypes, getPrintLevel, getPriorProbabilities, getRandomObject, getResponseColumnIndex, getResponseVariableAverage, getResponseVariableMostFrequentClass, getResponseVariableType, getTotalWeight, getVariableType, getWeights, getXY, isConstantSeries, isMustFitModel, isUserFixedNClasses, predict, setClassCounts, setClassLabels, setClassProbabilities, setCostMatrix, setMaxNumberOfCategories, setMaxNumberOfIterations, setMustFitModel, setNumberOfClasses, setPredictorIndex, setPredictorTypes, setPrintLevel, setPriorProbabilities, setRandomObject, setResponseColumnIndex, setTrainingData, setVariableType, setWeights

Methods inherited from class java.lang.Object

equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

SupportVectorMachine

public SupportVectorMachine(double[][] xy,
                            int responseColumnIndex,
                            PredictiveModel.VariableType[] varType)

Constructs a support vector machine for a single response variable and multiple predictor variables.

Parameters:

xy - a double matrix containing the training data and associated response values

responseColumnIndex - an int specifying the column index of the response variable

varType - a PredictiveModel.VariableType array of length equal to xy[0].length containing the type of each variable

SupportVectorMachine

public SupportVectorMachine(double[][] xy,
                            int responseColumnIndex,
                            PredictiveModel.VariableType[] varType,
                            Kernel k)

Constructs a support vector machine for a single response variable and multiple predictor variables.

Parameters:

xy - a double matrix containing the training data and associated response values

responseColumnIndex - an int specifying the column index of the response variable

varType - a PredictiveModel.VariableType array of length equal to xy[0].length containing the type of each variable

k - a Kernel, the kernel function

Method Detail

fitModel

public void fitModel()
              throws PredictiveModel.PredictiveModelException

Fits the model to the training data, i.e, trains the support vector machine.

Overrides:

fitModel in class PredictiveModel

Throws:

PredictiveModel.PredictiveModelException - an exception has occurred in the com.imsl.datamining.PredictiveModel. Superclass exceptions should be considered such as com.imsl.datamining.PredictiveModel.StateChangeException, com.imsl.datamining.PredictiveModel.SumOfProbabilitiesNotOneException, and com.imsl.datamining.supportvectormachine.SupportVectorMachine.ReflectiveOperationException.

setConfiguration

protected void setConfiguration(PredictiveModel pm)

Sets the configuration of PredictiveModel to that of the input model.

Note that the input PredictiveModel object must be the same subclass of PredictiveModel as is this instance.

Specified by:

setConfiguration in class PredictiveModel

Parameters:

pm - a PredictiveModel object which is to have its attributes duplicated in this instance

Throws:

CloneNotSupportedException - a java.lang.CloneNotSupportedException has occurred. The original exception has been added to the SupportVectorMachine.CloneNotSupportedException as a suppressed exception.

Default: The class uses its default configuration as described in the different methods.

predict

public double[] predict()
                 throws PredictiveModel.SumOfProbabilitiesNotOneException

Returns the predicted values on the training data, i.e., returns the fitted values.

Specified by:

predict in class PredictiveModel

Returns:

a double array containing the fitted values on the training data

Throws:

SumOfProbabilitiesNotOneException - the sum of probabilities is not approximately one

PredictiveModel.SumOfProbabilitiesNotOneException

predict

public double[] predict(double[][] testData)
                 throws PredictiveModel.SumOfProbabilitiesNotOneException

Returns the predicted values on the input test data.

Specified by:

predict in class PredictiveModel

Parameters:

testData - a double matrix containing test data

Note: testData must have the same number of columns as xy and the columns must be in the same arrangement as in xy.

Returns:

a double array containing the predicted values

Throws:

SumOfProbabilitiesNotOneException - the sum of probabilities is not approximately one

PredictiveModel.SumOfProbabilitiesNotOneException

setKernel

public void setKernel(Kernel kernel)

Sets the kernel to be used in the optimization.

See the class Kernel for details.

Parameters:

kernel - an instance of class Kernel

Default: kernel=RadialBasisKernel

getKernel

protected Kernel getKernel()

Returns the kernel object being used in the optimization.

Returns:

a Kernel object

getConvergenceTolerance

public double getConvergenceTolerance()

Returns the convergence tolerance.

Returns:

a double, the tolerance value

setConvergenceTolerance

public void setConvergenceTolerance(double eps)

Sets the convergence tolerance.

Parameters:

eps - a double, the tolerance value

Default: eps=0.001

setNuFormulation

public void setNuFormulation(boolean nuFormulation)

Sets the boolean to perform the \(\nu\)-formulation of the optimization problem. When true, the \(\nu\)-formulation of the optimization problem is solved. When false, the standard formulation is solved.

The \(\nu\)-support vector classification (\(\nu\)-SVC) algorithm presents a new parameter \(\nu \in (0,1] \) which acts as an upper bound on the fraction of training errors and a lower bound on the fraction of support vectors.

The primal optimization problem for the binary variable \(y \in \{1,-1\} \) is

$$\min_ {w, b, \xi, \rho} \frac{1}{2} w^Tw-\nu\rho+\frac{1}{l}\sum_{i=1}^{l} \xi _i$$

$$\text{subject to} \, y_i(w^T \phi (x_i)+b) \geq \rho - \xi _i,$$

$$ \xi _i \geq 0, i=1, \, \ldots \,,\, l\,,\,\rho \geq 0$$

Similar to \(\nu\)-SVC, in \(\nu\)-support vector regression (\(\nu\)-SVR) the parameter \(\nu \in (0,1] \) controls the number of support vectors. The \(\nu\)-SVR primal problem is

$$\min_ {w, b, \xi, \xi^*, \varepsilon} \frac{1}{2} w^Tw+C(\nu\varepsilon + \frac{1}{l}\sum_{i=1}^{l} (\xi _i + {\xi _i}^*))$$

$$\text{subject to} \, \, (w^T \phi (x_i)+b)-z_i \leq \varepsilon +\xi _i,$$

$$z_i - (w^T \phi (x_i)+b) \leq \varepsilon +{\xi _i}^*,$$

$$ \xi_i,\xi_i^*\ge 0,i=1,\;\ldots,\;l,\varepsilon\ge 0 $$

Parameters:

nuFormulation - a boolean, giving the value for the flag

Default: nuFormulation=false

isNuFormulation

public boolean isNuFormulation()

Returns the boolean to perform the \(\nu\)-formulation of the optimization problem. When true, the \(\nu\)-formulation of the optimization problem is solved. When false, the standard or default formulation is solved.

Returns:

a boolean, the current value of the flag

getNuParameter

public double getNuParameter()

Returns the value of the \(\nu\) parameter.

Returns:

a double, the current value of the \(\nu\) parameter

setNuParameter

public void setNuParameter(double nu)

Sets the value of \(\nu\) in the \(\nu\)-formulation of the optimization problem.

Parameters:

nu - a double in (0,1], the \(\nu\) parameter

Default: nu=0.5

getWorkingArraySize

public double getWorkingArraySize()

Returns the setting for the work array size.

Returns:

a double, the number of megabytes allocated to the work array used in the decomposition method

setWorkArraySize

public void setWorkArraySize(double workSize)

Sets the work array size.

A larger work array size can reduce the computational time of the decomposition method.

Parameters:

workSize - a double, the number of megabytes allocated for the array used during the decomposition method. workSize must be greater than 0.

Default: workSize = 100.0

getKernelParameters

public double[] getKernelParameters()

Returns the kernel parameters. See documentation for each Kernel class for specifications of the parameters.

Returns:

a double array containing the kernel parameter values

setKernelParameters

public void setKernelParameters(double[] kParams)

Sets the kernel parameters. See documentation for each Kernel class for specifications of the parameters.

Parameters:

kParams - a double array containing the kernel parameter values

isProbability

public boolean isProbability()

Returns the boolean to calculate probability estimates.

Returns:

a boolean, indicating whether or not probability estimates for each class level should be computed

setProbability

public void setProbability(boolean probability)

Sets the boolean to calculate probability estimates.

In classification problems, when the flag is true, class probabilities are estimated during the training procedure. For regression problems, when the flag is true, distributional parameters will be estimated that allow probability inferences for the response variable. When false, probability information is not calculated.

Parameters:

probability - a boolean, indicating whether or not to calculate probability estimates

Default: probability=false

getRegularizationParameter

public double getRegularizationParameter()

Returns the value of the regularization parameter, C.

Returns:

a double, the value of the regularization parameter

setRegularizationParameter

public void setRegularizationParameter(double C)

Sets the regularization parameter, C.

Parameters:

C - a double, greater than 0.0

Default: C=1.0

isShrinking

public boolean isShrinking()

Returns the boolean to perform shrinking during optimization. When true, the optimization performs shrinking.

Returns:

a boolean, the value of the shrinking flag

setShrinking

public void setShrinking(boolean shrinking)

Sets the boolean to perform shrinking during optimization.

The shrinking technique tries to identify and remove some bounded elements during the application of the SMO (sequential minimal optimization) algorithm, so that a smaller optimization problem is solved.

Parameters:

shrinking - a boolean, indicating whether or not shrinking should be performed during the optimization algorithm

Default: shrinking=true

getModel

protected SVModel getModel()

Returns the model object.

Returns:

an SVModel object

optimize

protected abstract SVModel optimize(DataNode[][] x,
                                    double[] y,
                                    double[] w,
                                    int len,
                                    Kernel kernel)
                             throws NoSuchMethodException,
                                    InstantiationException,
                                    IllegalAccessException,
                                    InvocationTargetException

Abstract method to perform the support vector machine optimization.

Parameters:

x - a DataNode matrix containing the attribute data

y - a double array containing the response variable

len - an int, the total possible number of support vectors

w - a double array containing the observation weights

kernel - an Kernel object

Returns:

an SVModel structure containing the fitted model

Throws:

NoSuchMethodException - thrown when a particular method cannot be found

InstantiationException - thrown when an application tries to create an instance of a class using the newInstance method in class Class, but the specified class object cannot be instantiated

IllegalAccessException - thrown when an application tries to reflectively create an instance (other than an array), set or get a field, or invoke a method, but the currently executing method does not have access to the definition of the specified class, field, method or constructor

InvocationTargetException - a checked exception that wraps an exception thrown by an invoked method or constructor

predictValues

protected abstract double[] predictValues(SVModel model,
                                          double[][] attributeData)
                                   throws PredictiveModel.SumOfProbabilitiesNotOneException

Abstract method for generating the predicted values using the fitted support vector machine model.

Parameters:

model - a fitted SVModel object

attributeData - a double matrix containing the attribute (or predictor) data

Returns:

a double array containing the predictions for each row in the input attribute data

Throws:

PredictiveModel.SumOfProbabilitiesNotOneException - the sum of probabilities is not approximately one