all-MiniLM-L6-v2
Jump to navigation
Jump to search
An all-MiniLM-L6-v2 is a SentenceTransformers pre-trained model (pre-trained sentence embedding model for SentenceTransformers) that maps sentences and paragraphs to a 384-dimensional sentence embedding.
- Context:
- It can (typically) have Model Size of: ~80MB.
- It can (typically) produce embeddings in milliseconds, enabling real-time applications.
- It can (typically) produce 384-dimensional dense vectors.
- It can (typically) handle sequences up to 256 tokens, optimizing for both short and moderate-length texts.
- It can support usage both with and without the sentence-transformers package, providing flexibility in implementation.
- It can be trained using a self-supervised contrastive learning objective on a dataset comprising over 1 billion sentence pairs.
- It can be fine-tuned from the pretrained microsoft/mpnet-base model (to enhance its sentence embedding capabilities).
- It can have been developed during a community week using JAX/Flax for NLP & CV.
- ...
- Example(s):
- ...
- Counter-Example(s):
- See: Contrastive Learning Objective, Dense Vector Space, Sentence Pair Encoding.
References
2024
- https://www.sbert.net/docs/pretrained_models.html
- QUOTE: The all-* models were trained on all available training data (more than 1 billion training pairs) and are designed as general purpose models. The all-mpnet-base-v2 model provides the best quality, while all-MiniLM-L6-v2 is 5 times faster and still offers good quality.
2024
- https://huggingface.co/sentence-transformers/all-MiniLM-L6-v2
- QUOTE: This is a sentence-transformers model: It maps sentences & paragraphs to a 384 dimensional dense vector space and can be used for tasks like clustering or semantic search.
- NOTES:
- It represents a Sentence Transformers project innovation, transforming NLP inputs into 384-dimensional vectors, enhancing clustering and semantic search.
- It necessitates the sentence-transformers library installation via pip for direct application, simplifying integration into NLP pipelines.
- It is compatible with the Hugging Face Transformers library, enabling broader usage across different NLP tasks without requiring the specific sentence-transformers installation.
- It has undergone thorough evaluation via the Sentence Embeddings Benchmark, demonstrating its efficacy in a variety of NLP tasks.
- It originated from a collaborative effort during a Community Week, utilizing JAX/Flax and a base MiniLM model, refined on an extensive dataset.
- It is crafted for generating semantically rich vectors from sentences and short paragraphs, aimed at tasks like Information Retrieval, Data clustering, and assessing Sentence similarity.
- It underwent fine-tuning with a contrastive learning objective on a diverse corpus, leveraging TPU technology for efficient training.