Sentence Embedding Algorithm

A Sentence Embedding Algorithm is a text-item embedding algorithm that can be implemented by a sentence embedding system to solve a sentence embedding task.

• AKA: Sentence Encoding Method.
• Context:
  • It can (typically) transform individual sentences into fixed-length vectors in a high-dimensional space.
  • It can (typically) leverage Neural Network Architectures, such as Recurrent Neural Networks (RNNs), Convolutional Neural Networks (CNNs), or Transformer models, for generating embeddings.
  • It can (often) be trained on large corpora of text data to capture a wide range of linguistic properties and nuances.
  • ...
• Example(s):
  • Sentence-BERT (SBERT) Algorithm, which adapts the BERT model using Siamese and triplet network structures for efficient sentence embedding.
  • Universal Sentence Encoder, which uses deep averaging network (DAN) and transformer architecture to generate sentence embeddings.
  • InferSent, a model trained on natural language inference data to derive universal sentence representations.
  • ...
• Counter-Example(s):
  • a Word Embedding Algorithm, which focuses on mapping individual words to vectors, without considering the broader sentence context.
  • a Document Embedding Algorithm, which generates document embeddings.
  • ...
• Counter-Example(s):
  • a Word Embedding Algorithm.
  • a Document Embedding Algorithm.
• See: Text Item, SentenceTransformers‎‎.

---

---

References

2017

• (Lin et al., 2017) ⇒ Zhouhan Lin, Minwei Feng, Cicero Nogueira dos Santos, Mo Yu, Bing Xiang, Bowen Zhou, and Yoshua Bengio. (2017). “A Structured Self-attentive Sentence Embedding.” In: Proceedings of the 5th International Conference on Learning Representations (ICRL-2017).

---