com.imsl.datamining.neural

Class LeastSquaresTrainer

java.lang.Object

com.imsl.datamining.neural.LeastSquaresTrainer

All Implemented Interfaces:

Trainer, Serializable

public class LeastSquaresTrainer
extends Object
implements Trainer, Serializable

Trains a FeedForwardNetwork using a Levenberg-Marquardt algorithm for minimizing a sum of squares error.

See Also:

NonlinLeastSquares, Serialized Form

Constructor Summary

Constructors

Constructor and Description
LeastSquaresTrainer() Creates a LeastSquaresTrainer.

Method Summary

Modifier and Type	Method and Description
protected Object	clone() Clones a copy of the trainer.
double[]	getErrorGradient() Returns the value of the gradient of the error function with respect to the weights.
int	getErrorStatus() Returns the error status from the trainer.
double	getErrorValue() Returns the final value of the error function.
protected void	setEpochNumber(int num) Sets the epoch number for the trainer.
void	setFalseConvergenceTolerance(double falseConvergenceTolerance) Set the false convergence tolerance.
void	setGradientTolerance(double gradientTolerance) Set the gradient tolerance.
void	setInitialTrustRegion(double initialTrustRegion) Sets the intial trust region.
void	setMaximumStepsize(double maximumStepsize) Sets the maximum step size.
void	setMaximumTrainingIterations(int maximumSolverIterations) Sets the maximum number of iterations used by the nonlinear least squares solver.
protected void	setParallelMode(ArrayList[] allLogRecords) Sets the trainer to be used in multi-threaded EpochTainer.
void	setRelativeTolerance(double relativeTolerance) Sets the relative tolerance.
void	setStepTolerance(double stepTolerance) Set the step tolerance used to step between weights.
void	train(Network network, double[][] xData, double[][] yData) Trains the neural network using supplied training patterns.

Methods inherited from class java.lang.Object

equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

LeastSquaresTrainer

public LeastSquaresTrainer()

Creates a LeastSquaresTrainer.

Method Detail

clone

protected Object clone()

Clones a copy of the trainer.

Overrides:

clone in class Object

setParallelMode

protected void setParallelMode(ArrayList[] allLogRecords)

Sets the trainer to be used in multi-threaded EpochTainer.

Parameters:

allLogRecords - An ArrayList array containing the log records.

setEpochNumber

protected void setEpochNumber(int num)

Sets the epoch number for the trainer.

Parameters:

num - An int array containing the epoch number.

setMaximumStepsize

public void setMaximumStepsize(double maximumStepsize)

Sets the maximum step size.

Parameters:

maximumStepsize - A nonnegative double value specifying the maximum allowable stepsize in the optimizer. Default: \(10^3 || w ||_2\), where w are the values of the weights in the network when training starts.

See Also:

NonlinLeastSquares.setMaximumStepsize(double)

setInitialTrustRegion

public void setInitialTrustRegion(double initialTrustRegion)

Sets the intial trust region.

Parameters:

initialTrustRegion - A double which specifies the initial trust region radius. Default: unlimited trust region.

See Also:

NonlinLeastSquares.setInitialTrustRegion(double)

setMaximumTrainingIterations

public void setMaximumTrainingIterations(int maximumSolverIterations)

Sets the maximum number of iterations used by the nonlinear least squares solver.

Parameters:

maximumSolverIterations - An int which specifies the maximum number of iterations to be used by the nonlinear least squares solver. Its default value is 1000.

See Also:

NonlinLeastSquares.setRelativeTolerance(double)

setRelativeTolerance

public void setRelativeTolerance(double relativeTolerance)

Sets the relative tolerance.

Parameters:

relativeTolerance - A double which specifies the relative error tolerance. It must be in the interval [0,1]. Its default value is 1.0e-20.

See Also:

NonlinLeastSquares.setRelativeTolerance(double)

setFalseConvergenceTolerance

public void setFalseConvergenceTolerance(double falseConvergenceTolerance)

Set the false convergence tolerance.

Parameters:

falseConvergenceTolerance - a double specifying the false convergence tolerance. Default: 1.0e-14.

See Also:

NonlinLeastSquares.setFalseConvergenceTolerance(double)

setGradientTolerance

public void setGradientTolerance(double gradientTolerance)

Set the gradient tolerance.

Parameters:

gradientTolerance - A double specifying the gradient tolerance. Default: 2.0e-5.

See Also:

MinUnconMultiVar.setGradientTolerance(double)

setStepTolerance

public void setStepTolerance(double stepTolerance)

Set the step tolerance used to step between weights.

Parameters:

stepTolerance - A double which specifies the scaled step tolerance to use when changing the weights. Default: 1.0e-5.

See Also:

NonlinLeastSquares.setStepTolerance(double)

train

public void train(Network network,
                  double[][] xData,
                  double[][] yData)

Trains the neural network using supplied training patterns.

Each row of xData and yData contains a training pattern. These number of rows in two arrays must be equal.

Specified by:

train in interface Trainer

Parameters:

network - The Network to be trained.

xData - A double matrix which contains the input training patterns. The number of columns in xData must equal the number of Nodes in the InputLayer.

yData - A double matrix which contains the output training patterns. The number of columns in yData must equal the number of Perceptrons in the OutputLayer.

getErrorValue

public double getErrorValue()

Returns the final value of the error function.

Specified by:

getErrorValue in interface Trainer

Returns:

A double containing the final value of the error function from the last training. Before training, NaN is returned.

getErrorGradient

public double[] getErrorGradient()

Returns the value of the gradient of the error function with respect to the weights.

Specified by:

getErrorGradient in interface Trainer

Returns:

A double array whose length is equal to the number of network weights, containing the value of the gradient of the error function with respect to the weights. Before training, null is returned.

getErrorStatus

public int getErrorStatus()

Returns the error status from the trainer.

Specified by:

getErrorStatus in interface Trainer

Returns:

An int which contains the error status. Zero indicates that no errors were encountered during training. Any non-zero value indicates that some error condition arose during training.

In many cases the trainer is able to recover from these conditions and produce a well-trained network.

Value	Meaning
0	All convergence tests were met.
1	Scaled step tolerance was satisfied. The current point may be an approximate local solution, or the algorithm is making very slow progress and is not near a solution, or StepTolerance is too big.
2	Scaled actual and predicted reductions in the function are less than or equal to the relative function convergence tolerance RelativeTolerance.
3	Iterates appear to be converging to a noncritical point. Incorrect gradient information, a discontinuous function, or stopping tolerances being too tight may be the cause.
4	Five consecutive steps with the maximum stepsize have been taken. Either the function is unbounded below, or has a finite asymptote in some direction, or the maximum stepsize is too small.
5	Too many iterations required