Neural Turing Machine (NTM)

From GM-RKB
Jump to navigation Jump to search

A Neural Turing Machine (NTM) is a Memory-Augmented Neural Network that combines a recurrent neural network with an external memory block.



References

2019

  • (Wikipedia, 2019) ⇒ https://en.wikipedia.org/wiki/Neural_Turing_machine Retrieved:2019-1-13.
    • A Neural Turing machine (NTMs) is a recurrent neural network model published by Alex Graves et. al. in 2014.[1] NTMs combine the fuzzy pattern matching capabilities of neural networks with the algorithmic power of programmable computers. An NTM has a neural network controller coupled to external memory resources, which it interacts with through attentional mechanisms. The memory interactions are differentiable end-to-end, making it possible to optimize them using gradient descent.[2] An NTM with a long short-term memory (LSTM) network controller can infer simple algorithms such as copying, sorting, and associative recall from input and output examples. They can infer algorithms from input and output examples alone.

      The authors of the original NTM paper did not publish the source code for their implementation . The first stable open-source implementation of a Neural Turing Machine was published in 2018 at the 27th International Conference on Artificial Neural Networks, receiving a best-paper award[3] [4][5]. Other open source implementations of NTMs exist but are not stable for production use.

The developers either report that the gradients of their implementation sometimes become NaN during training for unknown reasons and causing training to fail; report slow convergence;or do not report the speed of learning of their implementation at all.
Differentiable neural computers are an outgrowth of neural Turing machines, with attention mechanisms that control where the memory is active, and improved performance.

2018

2016

2016

2014a

2014b