Stacked Recurrent Neural Network

From GM-RKB
Jump to navigation Jump to search

A Stacked Recurrent Neural Network is a Stacked Neural Network that is a combination of recurrent neural networks.



References

2014

[math]\displaystyle{ h_{t}^{(l)}=f_{h}^{(l)}\left(h_{t}^{(l-1)}, h_{t-1}^{(l)}\right)=\phi_{h}\left(W_{l}^{T} h_{t-1}^{(l)}+U_{l}^{T} h_{t}^{(l-1)}\right) }[/math]

where, $h_{t}^{(l)}$ is the hidden state of the $l$-th level at time $t$. When $l = 1$, the state is computed using $x_t$ instead of $h_{t}^{(l-1)}$. The hidden states of all the levels are recursively computed from the bottom level $l= 1$.

Lambert S-RNN.png
Figure 2: Stacked Recurrent Neural Network Architecture.

2013

FIG2-Pascanu SNN.png
Figure 2: Illustrations of four different recurrent neural networks (RNN). (a) A conventional RNN. (b) Deep Transition (DT) RNN. (b*) DT-RNN with shortcut connections (c) Deep Transition, Deep Output (DOT) RNN. (d) Stacked RNN.

1996