Bridging the Question-Answer Gap in Retrieval-Augmented Generation: Hypothetical Prompt Embeddings
Domen Vake, Jernej Vičič, Aleksandar Tošić
cs.IR, cs.CL
Submitted: 2026-07-31
Comments: 10 pages, 8 figures, 5 tables. Published in IEEE Access
Journal ref: IEEE Access, vol. 13, pp. 129952-129961, 2025
DOI: 10.1109/ACCESS.2025.3589499
License: http://creativecommons.org/licenses/by/4.0/
The gist: Retrieval-Augmented Generation (RAG) systems synergize retrieval mechanisms with generative language models to enhance the accuracy and relevance of responses.
Terminology
Abstract
Retrieval-Augmented Generation (RAG) systems synergize retrieval mechanisms with generative language models to enhance the accuracy and relevance of responses. However, bridging the style gap between user queries and relevant information in document text remains a persistent challenge in retrieval-augmented systems, often addressed by runtime solutions (e.g., Hypothetical Document Embeddings (HyDE)) that attempt to improve alignment but introduce extra computational overhead at query time. To address these challenges, we propose Hypothetical Prompt Embeddings (HyPE), a framework that shifts the generation of hypothetical content from query time to the indexing phase. By precomputing multiple hypothetical prompts for each data chunk and embedding the chunk in place of the prompt, HyPE transforms retrieval into a question-question matching task, bypassing the need for runtime synthetic answer generation. This approach does not introduce latency but also strengthens the alignment between queries and relevant context. Our experimental results on six common datasets show that HyPE can improve retrieval context precision by up to 42 percentage points and claim recall by up to 45 percentage points, compared to standard approaches, while remaining compatible with re-ranking, multi-vector retrieval, query decomposition, and other RAG advancements
Sources
- Financial Report Chunking for Effective Retrieval Augmented Generation
- QA-GNN: Reasoning with Language Models and Knowledge Graphs for Question Answering
- Precise Zero-Shot Dense Retrieval without Relevance Labels
- Atlas: Few-shot Learning with Retrieval Augmented Language Models
- Document Expansion by Query Prediction
- Zero-shot Neural Passage Retrieval via Domain-targeted Synthetic Question Generation
- ARAGOG: Advanced RAG Output Grading
- A Comprehensive Survey of Retrieval-Augmented Generation (RAG): Evolution, Current Landscape and Future Directions
- A Survey on Knowledge-Oriented Retrieval-Augmented Generation
- Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks
- Dense Passage Retrieval for Open-Domain Question Answering
- RAGChecker: A Fine-grained Framework for Diagnosing Retrieval-Augmented Generation
- RAGBench: Explainable Benchmark for Retrieval-Augmented Generation Systems
Related papers
- The Price of Isolation: Estimating the Ecosystem Cost of Symmetric Two-Sided A/B Testing
- SCAR: Semantic Continuity-Aware Retrieval for Efficient Context Expansion in RAG
- MixLoRA-DSI: Dynamically Expandable Mixture-of-LoRA Experts for Rehearsal-Free Generative Retrieval over Dynamic Corpora
- QProtFuse: Query-Guided Layer Fusion for Retrieval-Augmented Protein Function Prediction
- DualView: Adaptive Local-Global Fusion for Multi-Hop Document Reranking
- How Do LLM-Generated Texts Impact Term-Based Retrieval Models?