PRIME: Pre-Retrieval Index Modeling Enhancement for Structured Document Retrieval-Real
cs.IR, cs.AI
Submitted: 2026-10-09
Updated: 2026-10-09
Comments: 9 pages, 2 figures. Manuscript completed July 2026. Supplementary material will be added in a future version
License: http://creativecommons.org/licenses/by-nc-nd/4.0/
The gist: High-precision retrieval over external knowledge bases is essential for LLM agents to operate effectively.
Abstract
High-precision retrieval over external knowledge bases is essential for LLM agents to operate effectively. However, existing Information Retrieval (IR) research primarily optimizes query-side retrievers (e.g., ColBERT, BGE) and generators (e.g., Self-RAG, GraphRAG), while reducing corpus-side representation to chunking. This leaves index-side quality an underexplored lever. We propose Pre-Retrieval Index Modeling & Enhancement (PRIME), an index-side optimization framework built on two complementary tracks: an LLM-driven Semantic Restructuring (SR) track that transforms long documents into structured, information-dense sections, and a deterministic Metadata Synthesis (MS) and Hierarchical Retrieval (HR) track that extracts semantic anchors and heading structure for coarse-to-fine section retrieval. Our evaluations show that deterministic metadata matches structure-aware chunking's retrieval quality while cutting embedding computation by 45.8 times and index vector count by 22.4 times. On top of these gains, Semantic Restructuring (SR) further improves Answer-F1 by 13.7% in full deployment, with characterized quality-fidelity-cost trade-offs. Across QASPER, BRIGHT, and NFCorpus, we further characterize three qualitative regimes that determine whether index-side optimization helps or degrades retrieval, while showing that these index-side effects are neutralized under a strong reranker.
Sources
- MS MARCO: A Human Generated MAchine Reading COmprehension Dataset
- Rethinking Chunk Size For Long-Document Retrieval: A Multi-Dataset Analysis
- Multi-view Content-aware Indexing for Long Document Retrieval
- From Local to Global: A Graph RAG Approach to Query-Focused Summarization
- Unsupervised Dense Information Retrieval with Contrastive Learning
- Passage Re-ranking with BERT
- Document Expansion by Query Prediction
- Beyond the Reranker: Do RAG Retrieval Enhancements Help Once a Strong Reranker Is Present?
- DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligence
Related papers
- The Price of Isolation: Estimating the Ecosystem Cost of Symmetric Two-Sided A/B Testing
- SCAR: Semantic Continuity-Aware Retrieval for Efficient Context Expansion in RAG
- MixLoRA-DSI: Dynamically Expandable Mixture-of-LoRA Experts for Rehearsal-Free Generative Retrieval over Dynamic Corpora
- How Do LLM-Generated Texts Impact Term-Based Retrieval Models?
- MemSyco-Bench: Benchmarking Sycophancy in Agent Memory
- From a Large-Scale Teacher to Efficient Multimodal Students for Search Relevance Prediction