Word Vector Space Model

A Word Vector Space Model is a text-item embedding model that is a lexical model by representing words within a word vector space.

• AKA: Word Embedding Model (WEM).
• Context:
  • It can be produced by a Word Vector Space Model Creation Task/Word Vectorizing Function Creation Task (that often accepts a corpus).
  • It can (typically) include a Word Vector Space Mapping Function.
  • It can range from (typically) being a Semantic Word Vector Model to being a Syntactic Word Vector Model.
  • It can range from being a Sparse Word Vector Space Model to being a Dense Word Vector Space Model.
  • It can range from being a Discrete Word Vector Space Model to being a Continuous Word Vector Space Model.
  • ...
• Example(s):
  • [math]\displaystyle{ f(\text{yellow})\rightarrow (\pi,2.3) }[/math]
  • Distributional Word Vectorizing Function, such as a word2vec Word Vector Space Model.
  • Bag-of-Words-based Word Vectorizing Function.
  • …
• Counter-Example(s):
  • a Sentence Vector Model w/ a Sentence Vectorizing Function.
  • a Word Classification Model.
  • a Document Vectorizing Function.
  • a Word Vector-Input Function.
  • an Image Vector Space Model.
• See: Text-Item Vectorization Function, Vector Distance Function.

---

---

References

2015

• (Vilnis & McCallum, 2015) ⇒ Luke Vilnis, and Andrew McCallum. (2015). “Word Representations via Gaussian Embedding.” In: arXiv preprint arXiv:1412.6623 submitted to ICRL 2015.
  • QUOTE: ... interest in learning compact distributed representations or embeddings for many machine learning tasks, including collaborative filtering (Koren et al., 2009), image retrieval (Weston et al., 2011), relation extraction (Riedel et al., 2013), word semantics and language modeling (Bengio et al., 2006; Mnih & Hinton, 2008; Mikolov et al., 2013), and many others. In these approaches input objects (such as images, relations or words) are mapped to dense vectors having lower-dimensionality than the cardinality of the inputs, with the goal that the geometry of his low-dimensional latent embedded space be smooth with respect to some measure of similarity in the target domain.

2011

• (Wikipedia, 2011) http://en.wikipedia.org/wiki/Vector_space_model
  • Vector space model (or term vector model) is an algebraic model for representing text documents (and any objects, in general) as vectors of identifiers, such as, for example, index terms. It is used in information filtering, information retrieval, indexing and relevancy rankings. Its first use was in the SMART Information Retrieval System.

1975

• (Salton et al., 1975) ⇒ Gerard M. Salton, A. Wong, and C. Yang. (1975). “A Vector Space Model for Automatic Indexing.” In: Communications of the ACM, 18(11). doi:10.1145/361219.361220.

---