Calculates classifications for trained multilayered feedforward neural networks.

The type double function is imsls_d_mlff_pattern_classification.

Pointer to an array of size n_patterns by n_classes containing the predicted class probabilities associated with each input pattern, where n_classes is the number of possible target classifications. n_classes = network‑>n_outputs for non-binary classification categories. For binary classification, n_classes = 2. This space can be released by using the imsls_free function.

Function imsls_f_mlff_pattern_classification calculates classification probabilities from a previously trained multilayered feedforward neural network using the same network structure and scaling applied during the training. The structure Imsls_f_NN_Network describes the network structure used to originally train the network. The weights, which are the key output from training, are used as input to this function. The weights are stored in the Imsls_f_NN_Network structure.

In addition, two two-dimensional arrays are used to describe the values of the nominal and continuous attributes that are to be used as network inputs for calculating classification probabilities. Optionally, it can also return the predicted classifications in pred_class. The predicted classification is the target class with the highest probability, class_prob.

Function imsls_f_mlff_pattern_classification returns classification probabilities for the network input patterns.

Neural network classification inputs consist of the following types of attributes:

The first data type contains the encoding of any nominal input attributes. If binary encoding is used, this encoding consists of creating columns of zeros and ones for each class value associated with every nominal attribute. The function imsls_f_unsupervised_nominal_filter can be used for this encoding.

When only one nominal attribute is used for input, then the number of binary encoded columns is equal to the number of classes for that attribute. If more nominal attributes appear in the data, then each nominal attribute is associated with several columns, one for each of its classes. Each column consists of zeros and ones. The column value is zero if that classification is not associated with this pattern; otherwise, it is equal to one if it is assigned to this pattern.

Consider an example with one nominal variable and two classes: male and female and the following five patterns: male, male, female, male, female. With binary encoding, the following 5 by 2 matrix is sent to the pattern classification to request classification probabilities for these patterns:

The second category of input attributes corresponds to continuous attributes. They are passed to this classification function via the floating point array continuous. The number of rows in this matrix is n_patterns, and the number of columns is n_continuous, corresponding to the number of continuous input attributes.

Ordinal input attributes, if used, are typically encoded to cumulative percentages. Since these are floating point values, they are placed into a column of the continuous array and n_continuous is set equal to the number of columns in this array.

In some cases, one of these types of input attributes may not exist. In that case, either n_nominal = 0 or n_continuous = 0 and their corresponding input matrix is ignored.

The configuration of the network consists of a description of the number of perceptrons for each layer, the number of hidden layers, the number of inputs and outputs, and a description of the linkages among the perceptrons. This description is passed into this training routine through the structure Imsls_f_NN_Network. See imsls_f_mlff_network. For binary problems there is only a single output since the probability P(class  = 0) is equal to 1-P(class  = 1). For other classification problems, however, n_outputs = n_classes and P(class = j) is equal to the classification probabilities in the j + 1 column of class_prob[].

Classification probabilities are calculated from the input attributes, network structure and weights provided in network.

Classification probabilities are returned in a two-dimensional array, class_prob, with n_patterns rows and n_classes columns. The values in the i-th column are estimated probabilities for the
class = (i-1).

Fisher’s (1936) Iris data is often used for benchmarking discriminant analysis and classification solutions. It is part of the IMSL data sets and consists of the following continuous input attributes and classification target:

Continuous Attributes – X1(sepal length), X2(sepal width), X3(petal length), and X4(petal width)

Classification Target (Iris Type) – Setosa, Versicolour or Virginica.

The input attributes were scaled to z-scores using imsls_f_scale_filter. The hidden layer contained only 2 perceptrons and the output layer consisted of three perceptrons, one for each classification target.

Example 2 for imsls_f_mlff_classification_trainer used the following network structure for the 150 patterns in these data:

imsls_f_mlff_classification_trainer found the following 19 weights for this network:

The association of these weights with the calculation of the potentials for each perceptron is described in the following table:

The potential calculations for each perceptron are activated using the assigned activation function. In this example, default activations were used, e.g. logistic for H1,1 and H1,2 and softmax for the output perceptrons H2,1, H2,2 and H2,3.

Note that in this case the network weights were retrieved from a file named iris_classfication.txt using imsls_f_mlff_network_read. This retrieves the trained network from mlff_classification_trainer described in Example 2. These were passed directly to imsls_f_mlff_pattern_classification in the Imsls_f_NN_Network structure.

The output for this example reproduces the 100% classification accuracy found during network training. For details, see Example 2 of imsls_f_mlff_classification_trainer.

Pattern classification is often used for pattern recognition, including playing simple games such as tic-tac-toe. The University of California at Irvine maintains a repository of data mining data, http://kdd.ics.uci.edu/. One consists of 958 patterns for board positions in tic-tac-toe donated by David Aha. See http://archive.ics.uci.edu/ml/datasets/Tic-Tac-Toe+Endgame for access to the actual data.

Each of the 958 patterns is described by nine nominal input attributes and one classification target. The nine nominal input attributes are the nine board positions in the game. Each has three classifications: X occupies the position, O occupies the position and vacant.

The target class is binary. A value of one indicates that the X player has one of eight possible wins in the next move. A value of zero indicates that this player does not have a winning position. 65.3% of the 958 patterns have a class = 1.

The nine nominal input attributes are mapped into 27 binary encoded columns, three for each of the nominal attributes. This makes a total of 27 input columns for the network. In this example, a neural network with one hidden layer containing ten perceptrons was found to provide 100% classification accuracy. This configuration is illustrated in the following figure.

All hidden layer perceptrons used the default activation, logistic, and since the classification target is binary only one perceptron with logistic activation is used to calculate the probability of a loss for X, i.e. P(class = 0). All logistic activations are calculated using the IMSLS_LOGISTIC_TABLE option, which can reduce Stage I training time. Since Stage II training is bypassed, this option must also be used with the imsls_f_mlff_pattern_classification routine. This is the only time this option is used. If Stage II training was part of the network training, the final network weights would have been calculated without using the logistic table to approximate the calculations.

This structure results in a network with 27× 8 + 8 + 9 = 233 weights. It is surprising that with this small a number of weights relative to the number of training patterns, the trained network achieves 100% classification accuracy.

Unlike Example 1 in which the network was trained previously and retrieved using imsls_f_mlff_network_read, this example first trains the network and then passes the network structure network into imsls_f_mlff_pattern_classification.

The output for this example demonstrates how imsls_f_mlff_pattern_classification reproduces the 100% classification accuracy found during network training.