Hopfield Network

Jump to: navigation, search

A Hopfield Network is a recurrent neural network with based on Lyapunov Functions.




  • (Kaushik, 2013) ⇒ Saroj Kaushik. (2013). “Artificial Neural Network - Lecture Model 22”. Course Material
    • QUOTE: A Hopfield network is a kind of recurrent network as output values are fed back to input in an undirected way.
      • It consists of a set of N connected neurons with weights which are symmetric and no unit is connected to itself.
      • There are no special input and output neurons.
      • The activation of a neuron is binary value decided by the sign of the weighted sum of the connections to it.
      • A threshold value for each neuron determines if it is a firing neuron.
      • A firing neuron is one that activates all neurons that are connected to it with a positive weight.
      • The input is simultaneously applied to all neurons, which then output to each other.
      • This process continues until a stable state is reached.



There are two popular forms of the model:
  • Binary neurons with discrete time, updated one at a time

    [math]V_j(t+1) = \begin{cases} 1, & \mbox{ if } \ \Sigma_k T_{jk}V_k(t) + I_j\gt 0 \\ 0, & \mbox{ otherwise } \end{cases} [/math]

  • Graded neurons with continuous time

    [math]dx_j/dt = -x_j/\tau + \Sigma_k T_{jk}g(x_k) + I_j\ .[/math]

  • [math]V_j[/math] denotes activity of the [math]j[/math]-th neuron.
  • [math]x_j[/math] is the mean internal potential of the neuron.
  • [math]I_j[/math] is direct input (e.g., sensory input or bias current) to the neuron.
  • [math]T_{jk}[/math] is the strength of synaptic input from neuron [math]k[/math] to neuron [math]j\ .[/math]
  • [math]g[/math] is a monotone function that converts internal potential into firing rate output of the neuron, i.e., [math]V_j=g(x_j)\ .[/math].


  • (Hopfield, 1982) ⇒ J. J. Hopfield. (1982). “Neural Networks and Physical systems with emergent collective computational abilities.” In: Proceedings of the National Academy of Sciences of the USA, vol. 79 no. 8.