We're excited to share our latest research on improving multi-fact retrieval in large language models (LLMs). Our paper identifies a "lost-in-the-middle" phenomenon in which LLMs struggle to maintain critical information throughout the generation process, leading to incomplete or inaccurate fact retrieval.
To tackle this, we introduce Find All Crucial Texts (FACT), an iterative retrieval method that refines context through successive rewriting rounds. FACT significantly boosts multi-fact retrieval performance, improving accuracy from ~70% to 99.4% in specific tasks like NIAH. While the improvements are less pronounced in general-purpose QA scenarios, FACT still enhances performance by over 20% on some datasets.
Our findings highlight the limitations of current LLMs in handling long-context retrieval and underscore the need for more robust strategies. The code will soon be available on GitHub, allowing the community to explore and build upon our work.
To tackle this, we introduce Find All Crucial Texts (FACT), an iterative retrieval method that refines context through successive rewriting rounds. FACT significantly boosts multi-fact retrieval performance, improving accuracy from ~70% to 99.4% in specific tasks like NIAH. While the improvements are less pronounced in general-purpose QA scenarios, FACT still enhances performance by over 20% on some datasets.
Our findings highlight the limitations of current LLMs in handling long-context retrieval and underscore the need for more robust strategies. The code will soon be available on GitHub, allowing the community to explore and build upon our work.