Package com.imsl.datamining.neural

Class FeedForwardNetwork

java.lang.Object
com.imsl.datamining.neural.Network
com.imsl.datamining.neural.FeedForwardNetwork
All Implemented Interfaces:
Serializable
public class FeedForwardNetwork extends Network
A representation of a feed forward neural network.

A Network contains an InputLayer, an OutputLayer and zero or more HiddenLayers. The null InputLayer and OutputLayer are automatically created by the Network constructor. The InputNodes are added using the getInputLayer().createInputs(nInputs) method. Output Perceptrons are added using the getOutputLayer().createPerceptrons(nOutputs), and HiddenLayers can be created using the createHiddenLayer().createPerceptrons(nPerceptrons) method.

The InputLayer contains InputNodes. The HiddenLayers and OutputLayers contain Perceptron nodes. These Nodes are created using factory methods in the Layers.

The Network also contains Links between Nodes. Links are created by methods in this class.

Each Link has a weight and gradient value. Each Perceptron node has a bias value. When the Network is trained, the weight and bias values are used as initial guesses. After the Network is trained the weight, gradient and bias values are set to the values computed by the training.

A feed forward network is a network in which links are only allowed from one layer to a following layer.

See Also:

  • Constructor Details

    • FeedForwardNetwork

      public FeedForwardNetwork()
      Creates a new instance of FeedForwardNetwork.

  • Method Details

    • getInputLayer

      public InputLayer getInputLayer()
      Returns the InputLayer.
      Specified by:
      getInputLayer in class Network
      Returns:
      The neural network InputLayer.

    • getOutputLayer

      public OutputLayer getOutputLayer()
      Returns the OutputLayer.
      Specified by:
      getOutputLayer in class Network
      Returns:
      The neural network OutputLayer.

    • createHiddenLayer

      public HiddenLayer createHiddenLayer()
      Creates a HiddenLayer.
      Specified by:
      createHiddenLayer in class Network
      Returns:
      A HiddenLayer object which specifies a neural network hidden layer.

    • getHiddenLayers

      public HiddenLayer[] getHiddenLayers()
      Returns the HiddenLayers in this network.
      Returns:
      An array of HiddenLayers in this network.

    • link

      public Link link(Node from, Node to)
      Establishes a Link between two Nodes. Any existing Link between these Nodes is removed.
      Parameters:
      from - The origination Node.
      to - The destination Node.
      Returns:
      A Link between the two Nodes.

    • link

      public Link link(Node from, Node to, double weight)
      Establishes a Link between two Nodes with a specified weight.
      Parameters:
      from - The origination Node.
      to - The destination Node.
      weight - A double which specifies the weight to be given the Link.
      Returns:
      A Link between the two Nodes.

    • linkAll

      public void linkAll(Layer from, Layer to)
      Link all of the Nodes in one Layer to all of the Nodes in another Layer.
      Parameters:
      from - The origination Layer.
      to - The destination Layer.

    • linkAll

      public void linkAll()
      For each Layer in the Network, link each Node in the Layer to each Node in the next Layer.

    • getLinks

      public Link[] getLinks()
      Return all of the Links in this Network.
      Specified by:
      getLinks in class Network
      Returns:
      An array of Links containing all of the Links in this Network.

    • findLinks

      public Link[] findLinks(Node to)
      Returns all of the Links to a given Node.
      Parameters:
      to - A Node who's Links are to be determined.
      Returns:
      An array of Links containing all of the Links to the given Node.

    • findLink

      public Link findLink(Node from, Node to)
      Returns the Link between two Nodes.
      Parameters:
      from - The origination Node.
      to - The destination Node.
      Returns:
      A Link between the two Nodes, or null if no such Link exists.

    • remove

      public void remove(Link link)
      Removes a Link from the network.
      Parameters:
      link - The Link deleted from the network.

    • getNumberOfInputs

      public int getNumberOfInputs()
      Returns the number of inputs to the Network.
      Specified by:
      getNumberOfInputs in class Network
      Returns:
      An int containing the number of inputs to the Network.

    • getNumberOfOutputs

      public int getNumberOfOutputs()
      Returns the number of outputs from the Network.
      Specified by:
      getNumberOfOutputs in class Network
      Returns:
      An int containing the number of outputs from the Network.

    • getNumberOfLinks

      public int getNumberOfLinks()
      Returns the number of Links in the Network.
      Specified by:
      getNumberOfLinks in class Network
      Returns:
      An int which contains the number of Links in the Network.

    • getPerceptrons

      public Perceptron[] getPerceptrons()
      Returns the Perceptrons in this Network.
      Specified by:
      getPerceptrons in class Network
      Returns:
      An array of Perceptrons in this network.

    • getWeights

      public double[] getWeights()
      Returns the weights for the Links in this network.
      Specified by:
      getWeights in class Network
      Returns:
      An array of doubles containing the weights. The array contains the weights for each Link followed by the Perceptron bias values. The Link weights are in the order in which the Links were created. The weight values are first, followed by the bias values in the HiddenLayers and then the bias values in the OutputLayer, and then by the order in which the Perceptrons were created.

    • setWeights

      public void setWeights(double[] weights)
      Sets the weights for the Links in this Network.
      Specified by:
      setWeights in class Network
      Parameters:
      weights - A double array containing the weights in the same order as getWeights().

    • setRandomWeights

      public void setRandomWeights(double[][] xData, Random random)
      Initializes network weights using random weights.

      The RandomWeights algorithm assigns equal weights to all Perceptrons, except those in the first Layer connected to the InputLayer. Like the EqualWeights algorithm, Perceptrons not in the first Layer are assigned weights 1/k, where k is the number of inputs connected to that Perceptron.

      For the first Layer Perceptron weights, they are initially assigned values from the uniform random distribution on the interval [-0.5, +0.5]. These are then scaled using the training patterns. The random weights for a Perceptron are divided by s, the standard deviation of the potential for that Perceptron calculated using the intial random values. Its bias value is set to -avg/s, where avg is the average potential for that Perceptron</code. This makes the average potential for the Perceptrons in this first Layer approximately 0 and its standard deviation equal to 1.

      This reduces the possibility of saturation during network training resulting from very large or small values for the Perceptron's potential. During training random noise is added to these intial values at each training stage. If the EpochTrainer is used, noise is added to these initial values at the start of each epoch.

      Parameters:
      xData - An input double matrix containing training patterns. The number of columns in xData must equal the number of Perceptrons in the InputLayer.
      random - A Random object.

    • setEqualWeights

      public void setEqualWeights(double[][] xData)
      Initializes network weights using equal weighting.

      The equal weights approach starts by assigning equal values to the inputs of each Perceptron. If a Perceptron has 4 inputs, then this method starts by assigning the value 1/4 to each of the perceptron's input weights. The bias weight is initially assigned a value of zero.

      The weights for the first Layer of Perceptrons, either the first HiddenLayer if the number of Layers is greater than 1 or the OutputLayer, are scaled using the training patterns. Scaling is accomplished by dividing the initial weights for the first Layer by the standard deviation, s, of the potential for that Perceptron. The bias weight is set to -avg/s, where avg is the average potential for that Perceptron. This makes the average potential for the Perceptrons in this first Layer approximately 0 and its standard deviation equal to 1.

      This reduces the possibility of saturation during network training resulting from very large or small values for the Perceptron's potential. During training random noise is added to these intial values at each training stage. If the EpochTrainer is used, noise is added to these initial values at the start of each epoch.

      Parameters:
      xData - An input double matrix containing training patterns. The number of columns in xData must equal the number of Perceptrons in the InputLayer.

    • getNumberOfWeights

      public int getNumberOfWeights()
      Returns the number of weights in the Network.
      Specified by:
      getNumberOfWeights in class Network
      Returns:
      An int which contains the number of weights in the Network.

    • validateLink

      protected void validateLink(Node from, Node to) throws IllegalArgumentException
      Checks that a Link between two Nodes is valid.

      In a FeedForwardNetwork a Link must be from a node in one Layer to a Node in a later Layer. Intermediate Layers can be skipped, but a Link cannot go backward.

      Parameters:
      from - The origination Node.
      to - The destination Node.
      Throws:
      IllegalArgumentException - is thrown if the Link is not valid

    • forecast

      public double[] forecast(double[] x)
      Computes a forecast using the Network.
      Specified by:
      forecast in class Network
      Parameters:
      x - A double array of values to which the Nodes in the InputLayer are to be set.
      Returns:
      A double array containing the values of the Nodes in the OutputLayer.

    • getForecastGradient

      public double[][] getForecastGradient(double[] xData)
      Returns the derivatives of the outputs with respect to the weights.
      Specified by:
      getForecastGradient in class Network
      Parameters:
      xData - A double array which specifies the input values at which the gradient is to be evaluated.
      Returns:
      A double array containing the gradient values. The value of gradient[i][j] is \(dy_i/dw_j\), where \(y_i\) is the i-th output and \(w_j\) is the j-th weight.