JMSLTM Numerical Library 6.1

com.imsl.datamining.neural
Class FeedForwardNetwork

java.lang.Object
  extended by com.imsl.datamining.neural.Network
      extended by com.imsl.datamining.neural.FeedForwardNetwork
All Implemented Interfaces:
Serializable

public class FeedForwardNetwork
extends Network

A representation of a feed forward neural network.

A Network contains an InputLayer, an OutputLayer and zero or more HiddenLayers. The null InputLayer and OutputLayer are automatically created by the Network constructor. The InputNodes are added using the getInputLayer().createInputs(nInputs) method. Output Perceptrons are added using the getOutputLayer().createPerceptrons(nOutputs), and HiddenLayers can be created using the createHiddenLayer().createPerceptrons(nPerceptrons) method.

The InputLayer contains InputNodes. The HiddenLayers and OutputLayers contain Perceptron nodes. These Nodes are created using factory methods in the Layers.

The Network also contains Links between Nodes. Links are created by methods in this class.

Each Link has a weight and gradient value. Each Perceptron node has a bias value. When the Network is trained, the weight and bias values are used as initial guesses. After the Network is trained the weight, gradient and bias values are set to the values computed by the training.

A feed forward network is a network in which links are only allowed from one layer to a following layer.

See Also:
Example 1, Serialized Form

Constructor Summary
FeedForwardNetwork()
          Creates a new instance of FeedForwardNetwork.
 
Method Summary
 HiddenLayer createHiddenLayer()
          Creates a HiddenLayer.
 Link findLink(Node from, Node to)
          Returns the Link between two Nodes.
 Link[] findLinks(Node to)
          Returns all of the Links to a given Node.
 double[] forecast(double[] x)
          Computes a forecast using the Network.
 double[][] getForecastGradient(double[] xData)
          Returns the derivatives of the outputs with respect to the weights.
 HiddenLayer[] getHiddenLayers()
          Returns the HiddenLayers in this network.
 InputLayer getInputLayer()
          Returns the InputLayer.
 Link[] getLinks()
          Return all of the Links in this Network.
 int getNumberOfInputs()
          Returns the number of inputs to the Network.
 int getNumberOfLinks()
          Returns the number of Links in the Network.
 int getNumberOfOutputs()
          Returns the number of outputs from the Network.
 int getNumberOfWeights()
          Returns the number of weights in the Network.
 OutputLayer getOutputLayer()
          Returns the OutputLayer.
 Perceptron[] getPerceptrons()
          Returns the Perceptrons in this Network.
 double[] getWeights()
          Returns the weights for the Links in this network.
 Link link(Node from, Node to)
          Establishes a Link between two Nodes.
 Link link(Node from, Node to, double weight)
          Establishes a Link between two Nodes with a specified weight.
 void linkAll()
          For each Layer in the Network, link each Node in the Layer to each Node in the next Layer.
 void linkAll(Layer from, Layer to)
          Link all of the Nodes in one Layer to all of the Nodes in another Layer.
 void remove(Link link)
          Removes a Link from the network.
 void setEqualWeights(double[][] xData)
          Initializes network weights using equal weighting.
 void setRandomWeights(double[][] xData, Random random)
          Initializes network weights using random weights.
 void setWeights(double[] weights)
          Sets the weights for the Links in this Network.
protected  void validateLink(Node from, Node to)
          Checks that a Link between two Nodes is valid.
 
Methods inherited from class com.imsl.datamining.neural.Network
computeStatistics
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 

Constructor Detail

FeedForwardNetwork

public FeedForwardNetwork()
Creates a new instance of FeedForwardNetwork.

Method Detail

createHiddenLayer

public HiddenLayer createHiddenLayer()
Creates a HiddenLayer.

Specified by:
createHiddenLayer in class Network
Returns:
A HiddenLayer object which specifies a neural network hidden layer.

findLink

public Link findLink(Node from,
                     Node to)
Returns the Link between two Nodes.

Parameters:
from - The origination Node.
to - The destination Node.
Returns:
A Link between the two Nodes, or null if no such Link exists.

findLinks

public Link[] findLinks(Node to)
Returns all of the Links to a given Node.

Parameters:
to - A Node who's Links are to be determined.
Returns:
An array of Links containing all of the Links to the given Node.

forecast

public double[] forecast(double[] x)
Computes a forecast using the Network.

Specified by:
forecast in class Network
Parameters:
x - A double array of values to which the Nodes in the InputLayer are to be set.
Returns:
A double array containing the values of the Nodes in the OutputLayer.

getForecastGradient

public double[][] getForecastGradient(double[] xData)
Returns the derivatives of the outputs with respect to the weights.

Specified by:
getForecastGradient in class Network
Parameters:
xData - A double array which specifies the input values at which the gradient is to be evaluated.
Returns:
A double array containing the gradient values. The value of gradient[i][j] is dy_i/dw_j, where y_i is the i-th output and w_j is the j-th weight.

getHiddenLayers

public HiddenLayer[] getHiddenLayers()
Returns the HiddenLayers in this network.

Returns:
An array of HiddenLayers in this network.

getInputLayer

public InputLayer getInputLayer()
Returns the InputLayer.

Specified by:
getInputLayer in class Network
Returns:
The neural network InputLayer.

getLinks

public Link[] getLinks()
Return all of the Links in this Network.

Specified by:
getLinks in class Network
Returns:
An array of Links containing all of the Links in this Network.

getNumberOfInputs

public int getNumberOfInputs()
Returns the number of inputs to the Network.

Specified by:
getNumberOfInputs in class Network
Returns:
An int containing the number of inputs to the Network.

getNumberOfLinks

public int getNumberOfLinks()
Returns the number of Links in the Network.

Specified by:
getNumberOfLinks in class Network
Returns:
An int which contains the number of Links in the Network.

getNumberOfOutputs

public int getNumberOfOutputs()
Returns the number of outputs from the Network.

Specified by:
getNumberOfOutputs in class Network
Returns:
An int containing the number of outputs from the Network.

getNumberOfWeights

public int getNumberOfWeights()
Returns the number of weights in the Network.

Specified by:
getNumberOfWeights in class Network
Returns:
An int which contains the number of weights in the Network.

getOutputLayer

public OutputLayer getOutputLayer()
Returns the OutputLayer.

Specified by:
getOutputLayer in class Network
Returns:
The neural network OutputLayer.

getPerceptrons

public Perceptron[] getPerceptrons()
Returns the Perceptrons in this Network.

Specified by:
getPerceptrons in class Network
Returns:
An array of Perceptrons in this network.

getWeights

public double[] getWeights()
Returns the weights for the Links in this network.

Specified by:
getWeights in class Network
Returns:
An array of doubles containing the weights. The array contains the weights for each Link followed by the Perceptron bias values. The Link weights are in the order in which the Links were created. The weight values are first, followed by the bias values in the HiddenLayers and then the bias values in the OutputLayer, and then by the order in which the Perceptrons were created.

link

public Link link(Node from,
                 Node to)
Establishes a Link between two Nodes. Any existing Link between these Nodes is removed.

Parameters:
from - The origination Node.
to - The destination Node.
Returns:
A Link between the two Nodes.

link

public Link link(Node from,
                 Node to,
                 double weight)
Establishes a Link between two Nodes with a specified weight.

Parameters:
from - The origination Node.
to - The destination Node.
weight - A double which specifies the weight to be given the Link.
Returns:
A Link between the two Nodes.

linkAll

public void linkAll()
For each Layer in the Network, link each Node in the Layer to each Node in the next Layer.


linkAll

public void linkAll(Layer from,
                    Layer to)
Link all of the Nodes in one Layer to all of the Nodes in another Layer.

Parameters:
from - The origination Layer.
to - The destination Layer.

remove

public void remove(Link link)
Removes a Link from the network.

Parameters:
link - The Link deleted from the network.

setEqualWeights

public void setEqualWeights(double[][] xData)
Initializes network weights using equal weighting.

The equal weights approach starts by assigning equal values to the inputs of each Perceptron. If a Perceptron has 4 inputs, then this method starts by assigning the value 1/4 to each of the perceptron's input weights. The bias weight is initially assigned a value of zero.

The weights for the first Layer of Perceptrons, either the first HiddenLayer if the number of Layers is greater than 1 or the OutputLayer, are scaled using the training patterns. Scaling is accomplished by dividing the initial weights for the first Layer by the standard deviation, s, of the potential for that Perceptron. The bias weight is set to -avg/s, where avg is the average potential for that Perceptron. This makes the average potential for the Perceptrons in this first Layer approximately 0 and its standard deviation equal to 1.

This reduces the possibility of saturation during network training resulting from very large or small values for the Perceptron's potential. During training random noise is added to these intial values at each training stage. If the EpochTrainer is used, noise is added to these initial values at the start of each epoch.

Parameters:
xData - An input double matrix containing training patterns. The number of columns in xData must equal the number of Perceptrons in the InputLayer.

setRandomWeights

public void setRandomWeights(double[][] xData,
                             Random random)
Initializes network weights using random weights.

The RandomWeights algorithm assigns equal weights to all Perceptrons, except those in the first Layer connected to the InputLayer. Like the EqualWeights algorithm, Perceptrons not in the first Layer are assigned weights 1/k, where k is the number of inputs connected to that Perceptron.

For the first Layer Perceptron weights, they are initially assigned values from the uniform random distribution on the interval [-0.5, +0.5]. These are then scaled using the training patterns. The random weights for a Perceptron are divided by s, the standard deviation of the potential for that Perceptron calculated using the intial random values. Its bias value is set to -avg/s, where avg is the average potential for that PerceptronPerceptrons in this first Layer approximately 0 and its standard deviation equal to 1.

This reduces the possibility of saturation during network training resulting from very large or small values for the Perceptron's potential. During training random noise is added to these intial values at each training stage. If the EpochTrainer is used, noise is added to these initial values at the start of each epoch.

Parameters:
xData - An input double matrix containing training patterns. The number of columns in xData must equal the number of Perceptrons in the InputLayer.
random - A Random object.

setWeights

public void setWeights(double[] weights)
Sets the weights for the Links in this Network.

Specified by:
setWeights in class Network
Parameters:
weights - A double array containing the weights in the same order as getWeights().

validateLink

protected void validateLink(Node from,
                            Node to)
                     throws IllegalArgumentException
Checks that a Link between two Nodes is valid.

In a FeedForwardNetwork a Link must be from a node in one Layer to a Node in a later Layer. Intermediate Layers can be skipped, but a Link cannot go backward.

Parameters:
from - The origination Node.
to - The destination Node.
Throws:
IllegalArgumentException - is thrown if the Link is not valid

JMSLTM Numerical Library 6.1

Copyright © 1970-2010 Visual Numerics, Inc.
Built July 30 2010.