Lecture in Natural Language Processing with Deep Learning - Stanford CS224N Ling284 (2017)

A Lecture in Natural Language Processing with Deep Learning - Stanford CS224N Ling284 (2017) is an academic lecture in natural language processing.

• Example(s):
  • (Manning & Socher, 2017i) ⇒ Christopher Manning, and Richard Socher. (2017). “Lecture 10 - Neural Machine Translation and Models with Attention.”
  • (Manning & Socher, 2017k) ⇒ Christopher Manning, and Richard Socher. (2017). “Lecture 11 - Further Topics in Neural Machine Translation and Recurrent Models.”
• See: NLP Course.

---

---

References

• https://cs224d.stanford.edu//lectures/

2017

• https://cs224d.stanford.edu//syllabus.html CS224d: Deep Learning for Natural Language Processing - Schedule and Syllabus

Event	Date	Description	Course Materials

Lecture Mar 29 Intro to NLP and Deep Learning Suggested Readings

   [Linear Algebra Review]
   [Probability Review]
   [Convex Optimization Review]
   [More Optimization (SGD) Review]
   [From Frequency to Meaning: Vector Space Models of Semantics]
[Lecture Notes 1]
[python tutorial] [slides]

Lecture Mar 31 Simple Word Vector representations

word2vec, GloVe Suggested Readings:

   [Distributed Representations of Words and Phrases and their Compositionality]
   [Efficient Estimation of Word Representations in Vector Space]
[slides]

Lecture Apr 5 Advanced word vector representations

language models, softmax, single layer networks Suggested Readings:

   [GloVe: Global Vectors for Word Representation]
   [Improving Word Representations via Global Context and Multiple Word Prototypes]
[Lecture Notes 2]

[slides]

Lecture Apr 7 Neural Networks and backpropagation -- for named entity recognition Suggested Readings

   [UFLDL tutorial]
   [Learning Representations by Backpropogating Errors]
[Lecture Notes 3]
[slides]

Lecture Apr 12 Project Advice, Neural Networks and Back-Prop (in full gory detail) Suggested Readings

   [Natural Language Processing (almost) from Scratch]
   [A Neural Network for Factoid Question Answering over Paragraphs]
   [Grounded Compositional Semantics for Finding and Describing Images with Sentences]
   [Deep Visual-Semantic Alignments for Generating Image Descriptions]
   [Recursive Deep Models for Semantic Compositionality Over a Sentiment Treebank]
[slides]

Lecture Apr 14 Practical tips

gradient checks, overfitting, regularization, activation functions, details Suggested Readings:

   [Practical recommendations for gradient-based training of deep architectures]
   [UFLDL page on gradient checking]
[slides]

Lecture Apr 19 Introduction to Tensorflow Suggested Readings

   [TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems]
[slides] [AWS Tutorial] [AWS Tutorial Supplementary] [AWS Tutorial Video]

Lecture Apr 21 Recurrent neural networks -- for language modeling and other tasks Suggested Readings

   [Recurrent neural network based language model]
   [Extensions of recurrent neural network language model]
   [Opinion Mining with Deep Recurrent Neural Networks]
[slides] [minimal net example (karpathy)] [vanishing grad example] [vanishing grad notebook]
[Lecture Notes 4]

Lecture Apr 26 GRUs and LSTMs -- for machine translation Suggested Readings

   [Long Short-Term Memory]
   [Gated Feedback Recurrent Neural Networks]
   [Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling]
[slides]

Lecture Apr 28 Recursive neural networks -- for parsing Suggested Readings

   [Parsing with Compositional Vector Grammars]
   [Subgradient Methods for Structured Prediction]
   [Parsing Natural Scenes and Natural Language with Recursive Neural Networks]
[Lecture Notes 5]
[slides]

Lecture May 3 Recursive neural networks -- for different tasks (e.g. sentiment analysis) Suggested Readings

   [Recursive Deep Models for Semantic Compositionality Over a Sentiment Treebank]
   [Dynamic Pooling and Unfolding Recursive Autoencoders for Paraphrase Detection]
   [Improved Semantic Representations From Tree-Structured Long Short-Term Memory Networks]

[slides]

Lecture May 12 Convolutional neural networks -- for sentence classification Suggested Readings

   [A Convolutional Neural Network for Modelling Sentences]

[slides]

Lecture May 17 Guest Lecture with Andrew Maas

Speech recognition Suggested Readings:

   [Deep Neural Networks for Acoustic Modeling in Speech Recognition]
[slides]

Lecture May 19 Guest Lecture with Thang Luong

Machine Translation Suggested Readings:

   [Achieving Open Vocabulary Neural Machine Translation with Hybrid Word-Character Models]
   [Addressing the Rare Word Problem in Neural Machine Translation]
   [Advances in natural language processing]
   [Neural machine translation by jointly learning to align and translate]
[slides]

Lecture May 24 Guest Lecture with Quoc Le

Seq2Seq and Large Scale DL Suggested Readings:

   [Sequence to Sequence with Neural Networks]
   [Neural Machine Translation by Jointly Learning to Align and Translate]
   [A Neural Conversation Model]
   [Neural Programmer: Include Latent Programs with Gradient Descent]
[slides]

Lecture May 26 The future of Deep Learning for NLP

Dynamic Memory Networks Suggested Readings:

   [Ask me anything: Dynamic Memory Networks for NLP]

---