Relevance Augmented Generation (RAG) Algorithm Technique

A Relevance Augmented Generation (RAG) Algorithm Technique is an LLM-based algorithm technique for RAG algorithms.

• Example(s):
  • RAG Contextual Compression.
  • RAG Relevance Reorganization.
  • RAG Self-Querying.
  • RAG FLARE.
  • GraphRAG.
  • ...
• See: Mads' CAS.

---

---

References

2023

• Claude 2
  • RAG Relevance reorganization is an advanced technique in retrieval augmented generation (RAG) architectures that uses a secondary smaller LLM to reorder retrieved documents and prioritize the most relevant information. Having the most relevant information appear first in the context helps optimize the efficiency and performance of the main LLM by front-loading the most critical information needed to generate an informed response.
    
  • RAG Contextual compression is another advanced RAG technique that uses a secondary smaller LLM to condense lengthy retrieved documents into more concise relevant context. This summarized content contains the key relevant facts without superfluous text, allowing for more condensed and efficient context to be passed to the primary LLM.
    
  • RAG Self-querying is a RAG technique that uses an additional LLM to interpret the original user query and transform it into a structured query format. For example, the self-querying LLM could convert a natural language question into an SQL query. This structured query can then pull specific information from databases and other systems, rather than relying solely on unstructured text retrieval. The self-querying LLM acts as an interpreter between the user's intent and a more optimized database query. This allows the capabilities of the retrieval system to be expanded through structured data lookup. Overall, self-querying adds more flexibility to RAG architectures by enabling LLM-powered query conversion to leverage both unstructured text and structured databases for context retrieval.
    
  • The RAG Framework for Locating and Acting upon Relevant External knowledge (FLARE) is an advanced technique used in retrieval augmented generation (RAG) systems to incorporate additional contextual information into summaries. It utilizes a secondary smaller LLM to analyze initially retrieved documents and identify key phrases that point to supplemental external knowledge sources. These phrases act as pointers for retrieving additional relevant documents to fill in any knowledge gaps. The newly retrieved external documents are passed into the summarization LLM along with the original documents to produce a summary enriched by incorporating the additional contextual information. Overall, FLARE provides a way for RAG systems to automatically detect missing knowledge in source documents, retrieve highly relevant supplementary information, and integrate this into more knowledge-complete summaries.

---