Creates a multilayered feedforward neural network.

The type double functions is imsls_d_mlff_network.

A multilayered feedforward network contains an input layer, an output layer and zero or more hidden layers. The input and output layers are created by the function imsls_f_mlff_network_init. The hidden layers are created by one or more calls to imsls_f_mlff_network with the keyword IMSLS_CREATE_HIDDEN_LAYER, where n_perceptrons specifies the number of perceptrons in the hidden layer.

The network also contains links or connections between nodes. Links are created by using one of the three optional arguments in the imsls_f_mlff_network function, IMSLS_LINK_ALL, IMSLS_LINK_LAYER, IMSLS_LINK_NODE. The most useful is the IMSLS_LINK_ALL, which connects every node in each layer to every node in the next layer. A feedforward network is a network in which links are only allowed from one layer to a following layer.

Each link has a weight and gradient value. Each perceptron node has a bias value. When the network is trained, the weight and bias values are used as initial guesses. After the network is trained using imsls_f_mlff_network_trainer, the weight, gradient and bias values are updated in the Imsls_f_NN_Network structure.

Each perceptron has an activation function g, and a bias, μ. The value of the percepton is given by g(Z), where g is the activation function and Z is the potential calculated using

where xi are the values of nodes input to this perceptron with weights wi.

All information for the network is stored in the structure called Imsls_f_NN_Network. (If the type is double, then the structure name is Imsls_d_NN_Network.) This structure describes the network that is trained by imsls_f_mlff_network_trainer.

where Imsls_NN_Layer is:

Imsls_NN_Link is:

and, Imsls_NN_Node is:

In particular, if network is a pointer to the structure of type Imsls_f_NN_Network , then:

Nodes are numbered starting at zero with the input nodes, followed by the hidden layer perceptrons and finally the output perceptrons.

Layers are numbered starting at zero with the input layer, followed by the hidden layers and finally the output layer. If there are no hidden layers, the output layer is numbered one.

Links are numbered starting at zero in the order the links were created. If the IMSLS_LINK_ALL option was used, the first link is the input link from the first input node to the first node in the next layer. The second link is the input link from the first input node to the second node in the next layer, continuing to the link from the last node in the next to last layer to the last node in the output layer. However, due to the possible variations in the order the links may be created, it is advised to initialize the weights using the imsls_f_initialize_weights routine or use the optional argument IMSLS_WEIGHT_INITIALIZATION_METHOD in functions imsls_f_mlff_network_trainer and imsls_f_mlff_classification_trainer. Alternatively, the weights can be initialized in the Imsls_f_NN_Network data structure. The following code is an example of how to initialize the network weights in an Imsls_f_NN_Network variable created with the name network:

The first for loop, j iterates through each perceptron in the network. Since input nodes are not perceptrons, they are excluded. The second for loop, k iterates through each of the perceptron’s input links, network->nodes[j].inLinks[k]. network‑>nodes[j].n_inLinks is the number of input links for network‑>nodes[j]. network‑>nodes[j].inLinks[k] contains the index of each input link to network‑>nodes[j] in the network‑>links array.

This example also illustrates how to set the weights based on the layer_id number. network‑>nodes[j].layer_id contains the layer identification number. This is used to set the weights for the first hidden layer to 0.5, the second hidden layer weights to 0.33 and all others to 0.25.

This example creates a single-layer feedforward network. The network inputs are directly connected to the output perceptrons using the IMSLS_LINK_ALL argument. The output perceptrons use the default linear activation function and default bias values of 0.0. The IMSLS_DISPLAY_NETWORK argument is used to show the default settings of the network.

This example creates a two-layer feedforward network with four inputs, one hidden layer with three perceptrons and two outputs.

Since the default activation function is linear for output and logistic for the hidden layers, to create a network that uses only linear activation you must specify the linear activation for each hidden layer in the network. This example demonstrates how to change the activation function and bias values for hidden and output layer perceptrons as shown in Figure 13.31 below.

This example creates a three-layer feedforward network with six input nodes and they are not all connected to every node in the first hidden layer.

Note also that the four perceptrons in the first hidden layer are not connected to every node in the second hidden layer, and the perceptrons in the second hidden layer are not all connected to the two outputs:

This network uses a total of nine perceptrons to produce two forecasts from six input attributes.

Links among the input nodes and perceptrons can be created using one of several approaches. If all inputs are connected to every perceptron in the first hidden layer, and if all perceptrons are connected to every perceptron in the following layer, which is a standard architecture for feed forward networks, then a call to the IMSLS_LINK_ALL method can be used to create these links.

However, this example does not use that standard configuration. Some links are missing. The keyword IMSLS_LINK_NODE can be used to construct individual links, or, an alternative approach is to first create all links and then remove those that are not needed. This example illustrates the latter approach.

Another approach is to use keywords IMSLS_LINK_NODE and IMSLS_LINK_LAYER to combine links between the two hidden layers, create individual links, and remove the links that are not needed. This example illustrates this approach: