# Hopfield Network

A Hopfield Network is a recurrent neural network with based on Lyapunov Functions.

## References

### 2013

• (Kaushik, 2013) ⇒ Saroj Kaushik. (2013). “Artificial Neural Network - Lecture Model 22”. Course Material
• QUOTE: A Hopfield network is a kind of recurrent network as output values are fed back to input in an undirected way.
• It consists of a set of N connected neurons with weights which are symmetric and no unit is connected to itself.
• There are no special input and output neurons.
• The activation of a neuron is binary value decided by the sign of the weighted sum of the connections to it.
• A threshold value for each neuron determines if it is a firing neuron.
• A firing neuron is one that activates all neurons that are connected to it with a positive weight.
• The input is simultaneously applied to all neurons, which then output to each other.
• This process continues until a stable state is reached.

### 2007

There are two popular forms of the model:
• Binary neurons with discrete time, updated one at a time

$V_j(t+1) = \begin{cases} 1, & \mbox{ if } \ \Sigma_k T_{jk}V_k(t) + I_j\gt 0 \\ 0, & \mbox{ otherwise } \end{cases}$

• Graded neurons with continuous time

$dx_j/dt = -x_j/\tau + \Sigma_k T_{jk}g(x_k) + I_j\ .$

Here,
• $V_j$ denotes activity of the $j$-th neuron.
• $x_j$ is the mean internal potential of the neuron.
• $I_j$ is direct input (e.g., sensory input or bias current) to the neuron.
• $T_{jk}$ is the strength of synaptic input from neuron $k$ to neuron $j\ .$
• $g$ is a monotone function that converts internal potential into firing rate output of the neuron, i.e., $V_j=g(x_j)\ .$.

### 1982

• (Hopfield, 1982) ⇒ J. J. Hopfield. (1982). “Neural Networks and Physical systems with emergent collective computational abilities.” In: Proceedings of the National Academy of Sciences of the USA, vol. 79 no. 8.