Attention-based Encoder-Decoder Network
Jump to navigation
Jump to search
A Attention-based Encoder-Decoder Network is a neural encoder-decoder network that includes an attention mechanism.
- Context:
- It can (typically) be used in tasks involving sequence-to-sequence (seq2seq) modeling, such as machine translation, speech recognition, and text summarization.
- It can (often) improve the performance of traditional seq2seq models by focusing on relevant parts of the input sequence for each step of the output sequence.
- It can include various forms of attention mechanisms, such as global attention, local attention, and multi-head attention.
- ...
- Example(s):
- the Transformer Neural Network, which uses a self-attention mechanism (Vaswani et al., 2017),
- the Neural Machine Translation by Jointly Learning to Align and Translate model (Bahdanau et al., 2015).
- the Pointer-Generator Seq2Seq Neural Network (See et al., 2017).
- ...
- Counter-Example(s):
- See: Sequence-to-Sequence Model, Neural Machine Translation, Recurrent Encoder-Decoder Neural Network, Seq2Seq with Attention Training Algorithm, Artificial Neural Network, Natural Language Processing Task, Language Model.