2018 ImplementingNeuralTuringMachine

• (Collier & Beel, 2018) ⇒ Mark Collier, and Joeran Beel. (2018). "Implementing Neural Turing Machines" (PDF). In: Proceedings of 27th International Conference on Artificial Neural Networks (ICANN). ISBN:978-3-030-01424-7. DOI:10.1007/978-3-030-01424-7_10. arXiv:1807.08518

Subject Headings: Neural Turing Machine; Memory-Augmented Neural Network, Memory-based Neural Network.

Cited By

• http://scholar.google.com/scholar?q=%222018%22+Implementing+Neural+Turing+Machines

Quotes

Abstract

Neural Turing Machines (NTMs) are an instance of Memory Augmented Neural Networks, a new class of recurrent neural networks which decouple computation from memory by introducing an external memory unit. NTMs have demonstrated superior performance over Long Short-Term Memory Cells in several sequence learning tasks. A number of open source implementations of NTMs exist but are unstable during training and/or fail to replicate the reported performance of NTMs. This paper presents the details of our successful implementation of a NTM. Our implementation learns to solve three sequential learning tasks from the original NTM paper. We find that the choice of memory contents initialization scheme is crucial in successfully implementing a NTM. Networks with memory contents initialized to small constant values converge on average 2 times faster than the next best memory contents initialization scheme.

References

;

	Author	volume	Date Value	title	type	journal	titleUrl	doi	note	year
2018 ImplementingNeuralTuringMachine	Joeran Beel Mark Collier			Implementing Neural Turing Machines				10.1007/978-3-030-01424-7_10		2018