Jordan Network

From GM-RKB
Jump to: navigation, search

A Jordan Network is a Simple Recurrent Network in which activations occur at the output layer, not at a hidden layer.



References

2018

  • (Wikipedia, 2018) ⇒ https://en.wikipedia.org/wiki/Recurrent_neural_network#Elman_networks_and_Jordan_networks Retrieved:2018-3-4.
    • Jordan networks are similar to Elman networks. The context units are fed from the output layer instead of the hidden layer. The context units in a Jordan network are also referred to as the state layer. They have a recurrent connection to themselves.

      Elman and Jordan networks are also known as "simple recurrent networks" (SRN).

      Elman network [1]:

      [math]\begin{align} h_t &= \sigma_h(W_{h} x_t + U_{h} h_{t-1} + b_h) \\ y_t &= \sigma_y(W_{y} h_t + b_y) \end{align} [/math]

      Jordan network [2]:

      [math] \begin{align} h_t &= \sigma_h(W_{h} x_t + U_{h} y_{t-1} + b_h) \\ y_t &= \sigma_y(W_{y} h_t + b_y) \end{align} [/math]

      Variables and functions

      • [math] x_t [/math] : input vector
      • [math] h_t [/math] : hidden layer vector
      • [math] y_t [/math] : output vector
      • [math] W [/math] , [math] U [/math] and [math] b [/math] : parameter matrices and vector
      • [math] \sigma_h [/math] and [math] \sigma_y [/math] : Activation functions
  1. Elman, Jeffrey L. (1990). “Finding Structure in Time". Cognitive Science. 14 (2): 179–211. doi:10.1016/0364-0213(90)90002-E.
  2. Jordan, Michael I. (1997-01-01). “Serial Order: A Parallel Distributed Processing Approach". Advances in Psychology. Neural-Network Models of Cognition. 121: 471–495. doi:10.1016/s0166-4115(97)80111-2. ISBN 9780444819314.

1997

ABSTRACT: A theory of learned sequential behavior is presented, with a focus on coarticulatory phenomena in speech. The theory is implemented as a recurrent parallel distributed processing network that is trained via a generalized error-correcting algorithm. The basic idea underlying the theory is that both serial order and coarticulatory overlap can be represented in terms of relative levels of activation in a network if a clear distinction is made between the state of the network and the output of the network.