Neural Network Hidden Layer

(Redirected from hidden layer)
Jump to navigation Jump to search

A Neural Network Hidden Layer is a neural network layer in between the Neural Network Input Layer and the Neural Network Output Layer.

[math]\displaystyle{ a^{(2)}_1 =g(\theta^{(1)}_{10} x_0+\theta^{(1)}_{11} x_1+\theta^{(1)}_{12} x_2+\theta^{(1)}_{13} x_3) }[/math]
[math]\displaystyle{ a^{(2)}_2 =g(\theta^{(1)}_{20} x_0+\theta^{(1)}_{21} x_1+\theta^{(1)}_{22} x_2+\theta^{(1)}_{23} x_3) }[/math]
[math]\displaystyle{ a^{(2)}_3 =g(\theta^{(1)}_{30} x_0+\theta^{(1)}_{31} x_1+\theta^{(1)}_{32} x_2+\theta^{(1)}_{33} x_3) }[/math]
[math]\displaystyle{ h_{\theta}(x)=a^{(3)}_1 =g(\theta^{(2)}_{10} a^{(2)}_0 +\theta^{(2)}_{11} a^{(2)}_1 +\theta^{(2)}_{12} a^{(2)}_2 +\theta^{(2)}_{13} a^{(2)}_3 ) }[/math]



Left: A 2-layer Neural Network (one hidden layer of 4 neurons (or units) and one output layer with 2 neurons), and three inputs.

Right:A 3-layer neural network with three inputs, two hidden layers of 4 neurons each and one output layer. Notice that in both cases there are connections (synapses) between neurons across layers, but not within a layer.

Naming conventions. Notice that when we say N-layer neural network, we do not count the input layer. Therefore, a single-layer neural network describes a network with no hidden layers (input directly mapped to output). In that sense, you can sometimes hear people say that logistic regression or SVMs are simply a special case of single-layer Neural Networks.


We classify the neural networks from their number of hidden layers and how they connect, for instance the network above have 2 hidden layers. Also if the neural network has/or not loops we can classify them as Recurrent or Feed-forward neural networks.

 Neural networks from more than 2 hidden layers can be considered a deep neural network.