Deep Neural Network (DNN) Training Algorithm

A Deep Neural Network (DNN) Training Algorithm is a multi-layer neural-network training algorithm that can be applied by a deep neural network training system to solve a deep neural network training task.

• Context:
  • It can (often) Minimize the Cross-Entropy of the Output Layer.
  • …
• Example(s):
  • a DNN Fine-Tuning Algorithm.
  • …
• Counter-Example(s):
  • a Single-Layer Neural Network Training Algorithm.
• See: Recurrent Neural Network Training Algorithm.

---

---

References

2016

• http://www.quora.com/What-are-the-likely-AI-advancements-in-the-next-5-to-10-years
  • Yann LeCun: There is a number of areas on which people are working hard and making promising advances:
    • deep learning combined with reasoning and planning.
    • deep model-based reinforcement learning (which involved unsupervised predictive learning)
    • recurrent neural nets augmented with differentiable memory modules (e.g. Memory Networks:
      • Memory Networks (FAIR).
      • Stack-Augmented RNN (FAIR).
      • Neural Turing Machine (DeepMind).
      • End-toEnd MemNN (FAIR/NYU)

---