Package com.imsl.datamining.supportvectormachine

Class SupportVectorMachine

java.lang.Object
com.imsl.datamining.PredictiveModel
com.imsl.datamining.supportvectormachine.SupportVectorMachine
All Implemented Interfaces:
Serializable, Cloneable
Direct Known Subclasses:
SVClassification, SVOneClass, SVRegression
public abstract class SupportVectorMachine extends PredictiveModel implements Serializable, Cloneable
Abstract class for generating a support vector machine.

Support vector machines are machine learning algorithms for classification, regression, and the so-called one class problem. This package contains SVClassification, SVRegression, and SVOneClass for each of these problems, respectively. Each formulation uses a kernel based optimization to find support vectors that optimally separate clouds of high dimensional data for the purpose of making predictions. SupportVectorMachine and all of the classes in this package are based on the LIBSVM package of Chang, Chih-Chung; Lin, Chih-Jen (2011).

See Also:

  • Constructor Details

    • SupportVectorMachine

      public SupportVectorMachine(double[][] xy, int responseColumnIndex, PredictiveModel.VariableType[] varType)
      Constructs a support vector machine for a single response variable and multiple predictor variables.
      Parameters:
      xy - a double matrix containing the training data and associated response values
      responseColumnIndex - an int specifying the column index of the response variable
      varType - a PredictiveModel.VariableType array of length equal to xy[0].length containing the type of each variable

    • SupportVectorMachine

      public SupportVectorMachine(double[][] xy, int responseColumnIndex, PredictiveModel.VariableType[] varType, Kernel k)
      Constructs a support vector machine for a single response variable and multiple predictor variables.
      Parameters:
      xy - a double matrix containing the training data and associated response values
      responseColumnIndex - an int specifying the column index of the response variable
      varType - a PredictiveModel.VariableType array of length equal to xy[0].length containing the type of each variable
      k - a Kernel, the kernel function

  • Method Details

    • fitModel

      public void fitModel() throws PredictiveModel.PredictiveModelException
      Fits the model to the training data, i.e, trains the support vector machine.
      Overrides:
      fitModel in class PredictiveModel
      Throws:
      PredictiveModel.PredictiveModelException - an exception has occurred in the com.imsl.datamining.PredictiveModel. Superclass exceptions should be considered such as com.imsl.datamining.PredictiveModel.StateChangeException, com.imsl.datamining.PredictiveModel.SumOfProbabilitiesNotOneException, and com.imsl.datamining.supportvectormachine.SupportVectorMachine.ReflectiveOperationException.

    • setConfiguration

      protected void setConfiguration(PredictiveModel pm)
      Sets the configuration of PredictiveModel to that of the input model.

      Note that the input PredictiveModel object must be the same subclass of PredictiveModel as is this instance.

      Specified by:
      setConfiguration in class PredictiveModel
      Parameters:
      pm - a PredictiveModel object which is to have its attributes duplicated in this instance
      Throws:
      PredictiveModel.CloneNotSupportedException - a java.lang.CloneNotSupportedException has occurred. The original exception has been added to the SupportVectorMachine.CloneNotSupportedException as a suppressed exception.

      Default: The class uses its default configuration as described in the different methods.

    • predict

      public double[] predict() throws PredictiveModel.SumOfProbabilitiesNotOneException
      Returns the predicted values on the training data, i.e., returns the fitted values.
      Specified by:
      predict in class PredictiveModel
      Returns:
      a double array containing the fitted values on the training data
      Throws:
      PredictiveModel.SumOfProbabilitiesNotOneException - the sum of probabilities is not approximately one

    • predict

      public double[] predict(double[][] testData) throws PredictiveModel.SumOfProbabilitiesNotOneException
      Returns the predicted values on the input test data.
      Specified by:
      predict in class PredictiveModel
      Parameters:
      testData - a double matrix containing test data

      Note: testData must have the same number of columns as xy and the columns must be in the same arrangement as in xy.

      Returns:
      a double array containing the predicted values
      Throws:
      PredictiveModel.SumOfProbabilitiesNotOneException - the sum of probabilities is not approximately one

    • setKernel

      public void setKernel(Kernel kernel)
      Sets the kernel to be used in the optimization.

      See the class Kernel for details.

      Parameters:
      kernel - an instance of class Kernel

      Default: kernel=RadialBasisKernel

    • getKernel

      protected Kernel getKernel()
      Returns the kernel object being used in the optimization.
      Returns:
      a Kernel object

    • getConvergenceTolerance

      public double getConvergenceTolerance()
      Returns the convergence tolerance.
      Returns:
      a double, the tolerance value

    • setConvergenceTolerance

      public void setConvergenceTolerance(double eps)
      Sets the convergence tolerance.
      Parameters:
      eps - a double, the tolerance value

      Default: eps=0.001

    • setNuFormulation

      public void setNuFormulation(boolean nuFormulation)
      Sets the boolean to perform the \(\nu\)-formulation of the optimization problem. When true, the \(\nu\)-formulation of the optimization problem is solved. When false, the standard formulation is solved.

      The \(\nu\)-support vector classification (\(\nu\)-SVC) algorithm presents a new parameter \(\nu \in (0,1] \) which acts as an upper bound on the fraction of training errors and a lower bound on the fraction of support vectors.

      The primal optimization problem for the binary variable \(y \in \{1,-1\} \) is

      $$\min_ {w, b, \xi, \rho} \frac{1}{2} w^Tw-\nu\rho+\frac{1}{l}\sum_{i=1}^{l} \xi _i$$

      $$\text{subject to} \, y_i(w^T \phi (x_i)+b) \geq \rho - \xi _i,$$

      $$ \xi _i \geq 0, i=1, \, \ldots \,,\, l\,,\,\rho \geq 0$$

      Similar to \(\nu\)-SVC, in \(\nu\)-support vector regression (\(\nu\)-SVR) the parameter \(\nu \in (0,1] \) controls the number of support vectors. The \(\nu\)-SVR primal problem is

      $$\min_ {w, b, \xi, \xi^*, \varepsilon} \frac{1}{2} w^Tw+C(\nu\varepsilon + \frac{1}{l}\sum_{i=1}^{l} (\xi _i + {\xi _i}^*))$$

      $$\text{subject to} \, \, (w^T \phi (x_i)+b)-z_i \leq \varepsilon +\xi _i,$$

      $$z_i - (w^T \phi (x_i)+b) \leq \varepsilon +{\xi _i}^*,$$

      $$ \xi_i,\xi_i^*\ge 0,i=1,\;\ldots,\;l,\varepsilon\ge 0 $$

      Parameters:
      nuFormulation - a boolean, giving the value for the flag

      Default: nuFormulation=false

    • isNuFormulation

      public boolean isNuFormulation()
      Returns the boolean to perform the \(\nu\)-formulation of the optimization problem. When true, the \(\nu\)-formulation of the optimization problem is solved. When false, the standard or default formulation is solved.
      Returns:
      a boolean, the current value of the flag

    • getNuParameter

      public double getNuParameter()
      Returns the value of the \(\nu\) parameter.
      Returns:
      a double, the current value of the \(\nu\) parameter

    • setNuParameter

      public void setNuParameter(double nu)
      Sets the value of \(\nu\) in the \(\nu\)-formulation of the optimization problem.
      Parameters:
      nu - a double in (0,1], the \(\nu\) parameter

      Default: nu=0.5

    • getWorkingArraySize

      public double getWorkingArraySize()
      Returns the setting for the work array size.
      Returns:
      a double, the number of megabytes allocated to the work array used in the decomposition method

    • setWorkArraySize

      public void setWorkArraySize(double workSize)
      Sets the work array size.

      A larger work array size can reduce the computational time of the decomposition method.

      Parameters:
      workSize - a double, the number of megabytes allocated for the array used during the decomposition method. workSize must be greater than 0.

      Default: workSize = 100.0

    • getKernelParameters

      public double[] getKernelParameters()
      Returns the kernel parameters. See documentation for each Kernel class for specifications of the parameters.
      Returns:
      a double array containing the kernel parameter values

    • setKernelParameters

      public void setKernelParameters(double[] kParams)
      Sets the kernel parameters. See documentation for each Kernel class for specifications of the parameters.
      Parameters:
      kParams - a double array containing the kernel parameter values

    • isProbability

      public boolean isProbability()
      Returns the boolean to calculate probability estimates.
      Returns:
      a boolean, indicating whether or not probability estimates for each class level should be computed

    • setProbability

      public void setProbability(boolean probability)
      Sets the boolean to calculate probability estimates.

      In classification problems, when the flag is true, class probabilities are estimated during the training procedure. For regression problems, when the flag is true, distributional parameters will be estimated that allow probability inferences for the response variable. When false, probability information is not calculated.

      Parameters:
      probability - a boolean, indicating whether or not to calculate probability estimates

      Default: probability=false

    • getRegularizationParameter

      public double getRegularizationParameter()
      Returns the value of the regularization parameter, C.
      Returns:
      a double, the value of the regularization parameter

    • setRegularizationParameter

      public void setRegularizationParameter(double C)
      Sets the regularization parameter, C.
      Parameters:
      C - a double, greater than 0.0

      Default: C=1.0

    • isShrinking

      public boolean isShrinking()
      Returns the boolean to perform shrinking during optimization. When true, the optimization performs shrinking.
      Returns:
      a boolean, the value of the shrinking flag

    • setShrinking

      public void setShrinking(boolean shrinking)
      Sets the boolean to perform shrinking during optimization.

      The shrinking technique tries to identify and remove some bounded elements during the application of the SMO (sequential minimal optimization) algorithm, so that a smaller optimization problem is solved.

      Parameters:
      shrinking - a boolean, indicating whether or not shrinking should be performed during the optimization algorithm

      Default: shrinking=true

    • getModel

      protected SVModel getModel()
      Returns the model object.
      Returns:
      an SVModel object

    • optimize

      protected abstract SVModel optimize(DataNode[][] x, double[] y, double[] w, int len, Kernel kernel) throws NoSuchMethodException, InstantiationException, IllegalAccessException, InvocationTargetException
      Abstract method to perform the support vector machine optimization.
      Parameters:
      x - a DataNode matrix containing the attribute data
      y - a double array containing the response variable
      len - an int, the total possible number of support vectors
      w - a double array containing the observation weights
      kernel - an Kernel object
      Returns:
      an SVModel structure containing the fitted model
      Throws:
      NoSuchMethodException - thrown when a particular method cannot be found
      InstantiationException - thrown when an application tries to create an instance of a class using the newInstance method in class Class, but the specified class object cannot be instantiated
      IllegalAccessException - thrown when an application tries to reflectively create an instance (other than an array), set or get a field, or invoke a method, but the currently executing method does not have access to the definition of the specified class, field, method or constructor
      InvocationTargetException - a checked exception that wraps an exception thrown by an invoked method or constructor

    • predictValues

      protected abstract double[] predictValues(SVModel model, double[][] attributeData) throws PredictiveModel.SumOfProbabilitiesNotOneException
      Abstract method for generating the predicted values using the fitted support vector machine model.
      Parameters:
      model - a fitted SVModel object
      attributeData - a double matrix containing the attribute (or predictor) data
      Returns:
      a double array containing the predictions for each row in the input attribute data
      Throws:
      PredictiveModel.SumOfProbabilitiesNotOneException - the sum of probabilities is not approximately one