Feed-Forward Neural Network (NNet) Training System

Jump to navigation Jump to search

A Feed-Forward Neural Network (NNet) Training System is a NNet training system that implements a feed-forward neural network training algorithm to solve a Feed-Forward Neural Network Training Task.



1. A stimulus-target pair is drawn from the training set.
2. The activity values for the units in the network are computed for all the units in the network in a forward fashion from input to output (Fig. 2a).
3. The network output values are compared to the target and a delta (δ) value is computed for each output unit based on the difference between the target and the actual output response value.
4. The deltas are propagated backward through the network using the same weights that were used to compute the activity values (Fig. 2b).
5. Each weight is updated by an amount proportional to the product of the downstream delta value and the upstream activity (Fig. 2c).
The procedure can be run either in an online mode or batch mode. In the online mode, the network parameters are updated for each stimulus-target pair. In the batch mode, the weight changes are computed and accumulated over several iterations without updating the weights until a large number (B) of stimulus-target pairs have been processed (often, the entire training set), at which the weights are updated by the accumulated amounts.


  • (PMML, 2017) ⇒ "PMML 4.0 - Neural Network Models" http://dmg.org/pmml/v4-0-1/NeuralNetwork.html#Example%20model Retrieved:2017-12-17
    • QUOTE: The description of neural network models assumes that the reader has a general knowledge of artificial neural network technology. A neural network has one or more input nodes and one or more neurons. Some neurons' outputs are the output of the network. The network is defined by the neurons and their connections, aka weights. All neurons are organized into layers; the sequence of layers defines the order in which the activations are computed. All output activations for neurons in some layer L are evaluated before computation proceeds to the next layer L+1. Note that this allows for recurrent networks where outputs of neurons in layer L+i can be used as input in layer L where L+i > L. The model does not define a specific evaluation order for neurons within a layer.

      Each neuron receives one or more input values, each coming via a network connection, and sends only one output value. All incoming connections for a certain neuron are contained in the corresponding Neuron element. Each connection Con of the element Neuron stores the ID of a node it comes from and the weight. A bias weight coefficient or a width of a radial basis function unit may be stored as an attribute of Neuron element.