Forward-Looking Active REtrieval Augmented Generation (FLARE) Algorithm
(Redirected from RAG FLARE)
Jump to navigation
Jump to search
A Forward-Looking Active REtrieval Augmented Generation (FLARE) Algorithm is a RAG algorithm technique that iteratively expands the context window information/
References
2023
- Claude 2
- QUOTE: The Framework for Locating and Acting upon Relevant External knowledge (FLARE) is an advanced technique used in retrieval augmented generation (RAG) systems to incorporate additional contextual information into summaries. It utilizes a secondary smaller LLM to analyze initially retrieved documents and identify key phrases that point to supplemental external knowledge sources. These phrases act as pointers for retrieving additional relevant documents to fill in any knowledge gaps. The newly retrieved external documents are passed into the summarization LLM along with the original documents to produce a summary enriched by incorporating the additional contextual information. Overall, FLARE provides a way for RAG systems to automatically detect missing knowledge in source documents, retrieve highly relevant supplementary information, and integrate this into more knowledge-complete summaries.
2023
- https://github.com/jzbjyb/FLARE
- QUOTE: FLARE is a generic retrieval-augmented generation method that actively decides when and what to retrieve using a prediction of the upcoming sentence to anticipate future content and utilize it as the query to retrieve relevant documents if it contains low-confidence tokens.
- QUOTE: FLARE is a generic retrieval-augmented generation method that actively decides when and what to retrieve using a prediction of the upcoming sentence to anticipate future content and utilize it as the query to retrieve relevant documents if it contains low-confidence tokens.
2023
- (Jiang, Xu et al., 2023) ⇒ Zhengbao Jiang, Frank F Xu, Luyu Gao, Zhiqing Sun, Qian Liu, Jane Dwivedi-Yu, Yiming Yang, Jamie Callan, and Graham Neubig. (2023). “Active Retrieval Augmented Generation.” In: arXiv preprint arXiv:2305.06983. doi:10.48550/arXiv.2305.06983
- ABSTRACT: Despite the remarkable ability of large language models (LMs) to comprehend and generate language, they have a tendency to hallucinate and create factually inaccurate output. Augmenting LMs by retrieving information from external knowledge resources is one promising solution. Most existing retrieval-augmented LMs employ a retrieve-and-generate setup that only retrieves information once based on the input. This is limiting, however, in more general scenarios involving generation of long texts, where continually gathering information throughout the generation process is essential. There have been some past efforts to retrieve information multiple times while generating outputs, which mostly retrieve documents at fixed intervals using the previous context as queries. In this work, we provide a generalized view of active retrieval augmented generation, methods that actively decide when and what to retrieve across the course of the generation. We propose Forward-Looking Active REtrieval augmented generation (FLARE), a generic retrieval-augmented generation method which iteratively uses a prediction of the upcoming sentence to anticipate future content, which is then utilized as a query to retrieve relevant documents to regenerate the sentence if it contains low-confidence tokens. We test FLARE along with baselines comprehensively over 4 long-form knowledge-intensive generation tasks/datasets. FLARE achieves superior or competitive performance on all tasks, demonstrating the effectiveness of our method. Code and datasets are available at this https URL.