Bi-Encoder Model

A Bi-Encoder Model is a neural retrieval model that independently computes vector representations for queries and documents enabling efficient similarity search.

• AKA: Dual-Encoder Model, Siamese Network Model, Dense Retrieval Model.
• Context:
  • It can typically generate Query Embeddings with neural encoder networks.
  • It can typically produce Document Embeddings with shared or separate encoders.
  • It can typically perform Approximate Nearest Neighbor Search through vector index structures.
  • It can often utilize Contrastive Learning for representation training.
  • It can often employ In-Batch Negatives for efficient negative sampling.
  • It can often apply Hard Negative Mining for discrimination improvement.
  • It can often integrate FAISS Library for scalable vector search.
  • It can range from being a Shallow Bi-Encoder Model to being a Deep Bi-Encoder Model, depending on its network depth.
  • It can range from being a Single-Tower Bi-Encoder Model to being a Dual-Tower Bi-Encoder Model, depending on its encoder architecture.
  • It can range from being a General Bi-Encoder Model to being a Domain-Specific Bi-Encoder Model, depending on its training specialization.
  • It can range from being a Monolingual Bi-Encoder Model to being a Multilingual Bi-Encoder Model, depending on its language coverage.
  • ...
• Examples:
  • Legal Bi-Encoder Models, such as:
    • Legal-BERT Bi-Encoder, trained on legal document pairs.
    • Sentence-BERT for Law, adapted for legal text similarity.
  • General-Purpose Bi-Encoder Models, such as:
    • Dense Passage Retrieval, ColBERT Model, ANCE Model.
  • ...
• Counter-Examples:
  • Cross-Encoder Model, which jointly processes query-document pairs.
  • Sparse Retrieval Model, which uses term-based representations.
  • Late Interaction Model, which allows token-level interactions.
• See: Cross-Encoder Model, Two-Stage Retrieval Architecture, Dense Retrieval Method, Contrastive Learning Technique for Legal Text, Vector Database, Sentence-BERT, Neural Information Retrieval, Neural Information Retrieval Model, Legal Language Embedding Model.

---