Single Hidden-Layer Neural Network
A Single Hidden-Layer Neural Network is an artificial neural network composed of one Neural Network Input Layer, one hidden neural network layer and one Neural Network Output Layer.
- AKA: 2-Layer Neural Network.
- Context:
- It can be trained by using a Single Hidden-Layer ANN Training System which implements a Single Hidden-Layer ANN Training Algorithm to solve Single Hidden-Layer ANN Training Task.
- It can (typically) be a Feedforward Neural Network,
- It can be graphically represented by
where [math]\{x_1,\cdots,x_p\}[/math] is the input vector, [math]\{y_1,\cdots,y_m\}[/math] is the output vector, [math]a_j[/math] is the hidden layer neurons activation, [math]w_{ij}[/math] and [math]w_{jk}[/math] are the input and output weight matrices associated with the synapses or network connections.
- It can be mathematically expressed as
[math]y_k=\sum_{j=1}^{n} w_{jk}*a_j+b_2, \quad a_j=f(\sum_{i=1}^{p} w_{ij}*x_i+b_1)[/math]
where [math]\{y_k\}_{k=1}^m[/math] is the output vector, [math]\{a_j\}_{j=1}^n[/math] is the hidden layer activation, [math]f[/math] is the activation function, [math]\{x_i\}_{i=1}^p[/math] the input vector, [math]w_{ij}[/math] is the input weight matrix,[math]\;w_{jk}[/math] is the output weight matrix, [math]b_1[/math] is the input layer bias and [math]b_2[/math] the hidden layer bias.
- Examples(s):
- Counter-Examples(s):
- See: Fully Connected Neural Network, Artificial Neural Network, Neural Network Topology, Single-layer Perceptron, Single-Layer ANN Training System, Activation Function, Artificial Neuron, Neural Network Layer.
References
2017
- (CS231n, 2017) ⇒ http://cs231n.github.io/neural-networks-1/#layers Retrieved: 2017-12-31
- QUOTE: Below are two example Neural Network topologies that use a stack of fully-connected layers:
Left: A 2-layer Neural Network (one hidden layer of 4 neurons (or units) and one output layer with 2 neurons), and three inputs. Right: A 3-layer neural network with three inputs, two hidden layers of 4 neurons each and one output layer. Notice that in both cases there are connections (synapses) between neurons across layers, but not within a layer. |
- Naming conventions. Notice that when we say N-layer neural network, we do not count the input layer. Therefore, a single-layer neural network describes a network with no hidden layers (input directly mapped to output). In that sense, you can sometimes hear people say that logistic regression or SVMs are simply a special case of single-layer Neural Networks. You may also hear these networks interchangeably referred to as “Artificial Neural Networks” (ANN) or “Multi-Layer Perceptrons” (MLP). Many people do not like the analogies between Neural Networks and real brains and prefer to refer to neurons as units.