public class GradientBoosting extends PredictiveModel implements Serializable, Cloneable
The idea behind boosting is to combine the outputs of relatively weak classifiers or predictive models to achieve iteratively better and better accuracy in either regression problems (the response variable is continuous) or classification problems (the response variable has two or more discrete values). This class implements the stochastic gradient tree boosting algorithm of Friedman, 1999. A sequence of decision trees is fit to random samples of the training data, iteratively re-weighted to minimize a specified loss function. In each iteration, pseudo-residuals are calculated based on a random sample from the original training set and the gradient of the loss function evaluated at values generated in the previous iteration. New base predictors are fit to the pseudo-residuals, and then a new prediction function is selected to minimize the loss-function, completing one iteration. The number of iterations is a parameter for the algorithm.
Gradient boosting is an ensemble method, but instead of using independent
trees, gradient boosting forms a sequence of trees, iteratively and
judiciously re-weighted to minimize prediction errors. In particular, the
decision tree at iteration
Modifier and Type | Class and Description |
---|---|
static class |
GradientBoosting.LossFunctionType
The loss function type as specified by the error measure.
|
PredictiveModel.PredictiveModelException, PredictiveModel.StateChangeException, PredictiveModel.SumOfProbabilitiesNotOneException, PredictiveModel.VariableType
Constructor and Description |
---|
GradientBoosting(double[][] xy,
int responseColumnIndex,
PredictiveModel.VariableType[] varType)
Constructs a
GradientBoosting object for a single response
variable and multiple predictor variables. |
GradientBoosting(PredictiveModel pm)
Constructs a
gradient boosting object. |
Modifier and Type | Method and Description |
---|---|
void |
fitModel()
Performs the gradient boosting on the training data.
|
double[][] |
getClassFittedValues()
Returns the fitted values for a categorical
response variable with two or more levels.
|
double[][] |
getClassProbabilities()
Returns the predicted probabilities on the training data for a
categorical response variable.
|
double[] |
getFittedValues()
Returns the fitted values for a continuous
response variable after gradient boosting.
|
int[] |
getIterationsArray()
Returns the array of different values for the number of iterations.
|
GradientBoosting.LossFunctionType |
getLossType()
Returns the current loss function type.
|
double |
getLossValue()
Returns the loss function value.
|
boolean |
getMissingTestYFlag()
Returns the flag that sets whether the test data is missing the response
variable data.
|
double[][] |
getMultinomialResponse()
Returns the multinomial representation of the response variable.
|
int |
getNumberOfIterations()
Returns the current setting for the number of iterations to use in the
gradient boosting algorithm.
|
double |
getSampleSizeProportion()
Returns the current setting of the sample size proportion.
|
double |
getShrinkageParameter()
Returns the current shrinkage parameter.
|
double[][] |
getTestClassFittedValues()
Returns the fitted values for a categorical
response variable with two or more levels on the test data.
|
double[][] |
getTestClassProbabilities()
Returns the predicted probabilities on the test data for a categorical
response variable.
|
double[] |
getTestFittedValues()
Returns the fitted values for a continuous
response variable after gradient boosting on the test data.
|
double |
getTestLossValue()
Returns the loss function value on the test data.
|
double[] |
predict()
Returns the predicted values on the training data.
|
double[] |
predict(double[][] testData)
Returns the predicted values on the input test data.
|
double[] |
predict(double[][] testData,
double[] testDataWeights)
Runs the gradient boosting on the training data and returns the predicted
values on the weighted test data.
|
void |
setIterationsArray(int[] iterationsArray)
Sets the array of different numbers of iterations.
|
void |
setLossFunctionType(GradientBoosting.LossFunctionType lossType)
Sets the loss function type for the gradient boosting algorithm.
|
void |
setMissingTestYFlag(boolean missingTestY)
Sets the flag determining whether the test data is missing the response
variable data.
|
void |
setNumberOfIterations(int numberOfIterations)
Sets the number of iterations.
|
void |
setSampleSizeProportion(double sampleSizeProportion)
Sets the sample size proportion.
|
void |
setShrinkageParameter(double shrinkageParameter)
Sets the value of the shrinkage parameter.
|
getClassCounts, getCostMatrix, getMaxNumberOfCategories, getNumberOfClasses, getNumberOfColumns, getNumberOfMissing, getNumberOfPredictors, getNumberOfRows, getNumberOfUniquePredictorValues, getPredictorIndexes, getPredictorTypes, getPrintLevel, getPriorProbabilities, getRandomObject, getResponseColumnIndex, getResponseVariableAverage, getResponseVariableMostFrequentClass, getResponseVariableType, getTotalWeight, getVariableType, getWeights, getXY, isMustFitModelFlag, isUserFixedNClasses, setClassCounts, setConfiguration, setCostMatrix, setFitModelFlag, setMaxNumberOfCategories, setNumberOfClasses, setPredictorIndex, setPredictorTypes, setPrintLevel, setPriorProbabilities, setRandomObject, setWeights
public GradientBoosting(double[][] xy, int responseColumnIndex, PredictiveModel.VariableType[] varType)
GradientBoosting
object for a single response
variable and multiple predictor variables.xy
- a double
matrix containing the training dataresponseColumnIndex
- an int
, the column index for the
response variablevarType
- a PredictiveModel.VariableType
array containing the type of each variablepublic GradientBoosting(PredictiveModel pm)
gradient boosting
object.pm
- the PredictiveModel
to serve as the base learner
Note: Currently only regression trees are supported as base learners.
public void fitModel() throws PredictiveModel.PredictiveModelException
fitModel
in class PredictiveModel
PredictiveModel.PredictiveModelException
- An
exception has occurred in the com.imsl.datamining.PredictiveModel.
Superclass exceptions should be considered such as
com.imsl.datamining.PredictiveModel.StateChangeException and
com.imsl.datamining.PredictiveModel.SumOfProbabilitiesNotOneException.public double[][] getClassFittedValues()
The underlying loss function is the binomial or multinomial deviance.
double
matrix containing the fitted values on the
training datapublic double[][] getClassProbabilities()
double
matrix containing the class probabilities
fit on the training data. The nClasses
-1.public double[] getFittedValues()
double
array containing the fitted values on the
training datapublic int[] getIterationsArray()
Different values for the number of iterations can be set and used in
cross validation. See setIterationsArray(int[])
.
int
array, containing the values for the number
of iterations parameterpublic GradientBoosting.LossFunctionType getLossType()
LossFunctionType
, the current setting of the loss
function typepublic double getLossValue()
double
, the loss function valuepublic boolean getMissingTestYFlag()
boolean
, the flag setting whether the test data
is missing the response variable valuespublic double[][] getMultinomialResponse()
is a matrix with the element at
nObservations
-1 and
nClasses
-1
Note: This representation is not available if the response has only 2 classes (binomial).
double
matrix containing the response in
multinomial representationpublic int getNumberOfIterations()
Different values for the number of iterations can be set and used in
cross validation. See setIterationsArray(int[])
.
int
, the current setting for the number of
iterationspublic double getSampleSizeProportion()
double
, the sample size proportionpublic double getShrinkageParameter()
double
, the value of shrinkage parameterpublic double[][] getTestClassFittedValues()
The underlying loss function is the binomial or multinomial deviance.
double
matrix containing the fitted values on the
test datapublic double[][] getTestClassProbabilities()
double
matrix containing the class probabilities
fit on the training data. The nClasses
-1.public double[] getTestFittedValues()
double
array containing the fitted values on the
test datapublic double getTestLossValue()
double
, the loss function valuepublic double[] predict() throws PredictiveModel.PredictiveModelException
predict
in class PredictiveModel
double
array containing the predicted values on
the training data, i.e., the fitted valuesPredictiveModel.PredictiveModelException
- An
exception has occurred in the com.imsl.datamining.PredictiveModel.
Superclass exceptions should be considered such as
com.imsl.datamining.PredictiveModel.StateChangeException and
com.imsl.datamining.PredictiveModel.SumOfProbabilitiesNotOneException.public double[] predict(double[][] testData) throws PredictiveModel.PredictiveModelException
predict
in class PredictiveModel
testData
- a double
matrix containing test data
Note: testData
must have the same number of columns and the
columns must be in the same arrangement as xy
.
double
array containing the predicted valuesPredictiveModel.PredictiveModelException
- An
exception has occurred in the com.imsl.datamining.PredictiveModel.
Superclass exceptions should be considered such as
com.imsl.datamining.PredictiveModel.StateChangeException and
com.imsl.datamining.PredictiveModel.SumOfProbabilitiesNotOneException.public double[] predict(double[][] testData, double[] testDataWeights) throws PredictiveModel.PredictiveModelException
predict
in class PredictiveModel
testData
- a double
matrix containing test data
Note:testData
must have the same number of columns and the
columns must be in the same arrangement as xy
.
testDataWeights
- a double
array containing weights for
each row of testData
double
array containing the predicted valuesPredictiveModel.PredictiveModelException
- An
exception has occurred in the com.imsl.datamining.PredictiveModel.
Superclass exceptions should be considered such as
com.imsl.datamining.PredictiveModel.StateChangeException and
com.imsl.datamining.PredictiveModel.SumOfProbabilitiesNotOneException.public void setIterationsArray(int[] iterationsArray)
The algorithm in gradient boosting iterates for a fixed number of times
and stops, rather than iterating until a convergence criteria is met. The
number of iterations is therefore a parameter in the model.
After setting the iterationsArray
to two or more values,
cross-validation can be used to help determine the best choice among the
values. By default, iterationsArray
contains the single
value {50}
.
iterationsArray
- an int
array containing the different
numbers of iterations
Default: iterationsArray
= {50}.
public void setLossFunctionType(GradientBoosting.LossFunctionType lossType)
lossType
- a LossFunctionType
, the desired loss
function type
Default:
lossType
=LossFunctionType.LEAST_SQUARES
public void setMissingTestYFlag(boolean missingTestY)
missingTestY
- a boolean
. When true
,
either the response variable data is all Double.NaN
or will
be treated as such.
Default: missingTestY
=false
public void setNumberOfIterations(int numberOfIterations)
numberOfIterations
- an int
, the number of iterations
Default: numberOfIterations
= 50. The
numberOfIterations
must be positive.
Note: This method resets iterationsArray
to an array of
length 1 containing numberOfIterations
.
public void setSampleSizeProportion(double sampleSizeProportion)
sampleSizeProportion
- a double
in the interval
specifying the desired sampling
proportion
Default: samplesSizeProportion
= 0.50. If
samplesSizeProportion
= 1.0, no sampling is performed.
public void setShrinkageParameter(double shrinkageParameter)
shrinkageParameter
- a double
in the interval
specifying the shrinkage parameter
Default: shrinkageParameter
=1.0 (no shrinkage)
Copyright © 1970-2015 Rogue Wave Software
Built October 13 2015.