What Would Fix This RAG Failure? Auditing Counterfactual Response with Paired Evidence Interventions

arXiv:2608.08944 · cs.IR, cs.LG · Submitted 2026-08-09 · Read on arXiv

Wenzhang Du

cs.IR, cs.LG

Submitted: 2026-08-09

Updated: 2026-08-11

Comments: 15 pages, 2 figures, 6 tables

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Terminology

• RAG — 21× in this paper · explained in 3 episodes
• Benchmark — 15× in this paper · explained in 2 episodes
• Accuracy — 14× in this paper · explained in 1 episode
• Regret — 12× in this paper · explained in 4 episodes
• AUROC — 6× in this paper · explained in 3 episodes
• Robustness — 4× in this paper · explained in 6 episodes
• Trace — 4× in this paper · explained in 2 episodes
• Calibration — 2× in this paper · explained in 9 episodes
• Generator — 2× in this paper · explained in 2 episodes
• Retrieval-Augmented Generation (RAG) — 2× in this paper · explained in 21 episodes
• Sufficiency — 2× in this paper · explained in 2 episodes
• Verifier — 2× in this paper · explained in 2 episodes
• Absorb — 1× in this paper · explained in 1 episode
• Attention Mechanism — 1× in this paper · explained in 3 episodes
• Augmentation — 1× in this paper · explained in 2 episodes
• Benchmarking — 1× in this paper · explained in 4 episodes
• Embeddings — 1× in this paper · explained in 2 episodes
• Generalization — 1× in this paper · explained in 14 episodes
• Papers — 1× in this paper · explained in 3 episodes
• Provenance — 1× in this paper · explained in 5 episodes
• Verification — 1× in this paper · explained in 2 episodes

Sources

• Benchmarking Large Language Models in Retrieval-Augmented Generation
• Gemma 3 Technical Report
• The Llama 3 Herd of Models
• Diagnosing and Repairing Factual Errors in RAG under Budget Constraints
• DeBERTaV3: Improving DeBERTa using ELECTRA-Style Pre-Training with Gradient-Disentangled Embedding Sharing
• CUE-R: Beyond the Final Answer in Retrieval-Augmented Generation
• Mistral 7B
• Doctor-RAG: A Failure-Aware Repair Framework for Agentic Retrieval-Augmented Generation
• When Knowledge Changes: Metamorphic Testing of RAG Systems with Mutations
• When Should LLMs Search? Counterfactual Supervision for Search Routing
• Evidence Interfaces Shape How Retrieval-Augmented Readers Use Support
• Bridge Evidence: Static Retrieval Utility Does Not Predict Causal Utility in Multi-Step Agentic Search
• Qwen3 Technical Report
• RAGChecker: A Fine-grained Framework for Diagnosing Retrieval-Augmented Generation
• Evidence Contextualization and Counterfactual Attribution for Conversational QA over Heterogeneous Data with RAG Systems
• Skill-RAG: Failure-State-Aware Retrieval Augmentation via Hidden-State Probing and Skill Routing
• Diagnosing Evidence Utilization in Long-Context and Retrieval-Augmented Language Models under Matched Evidence Conditions
• C-Pack: Packed Resources For General Chinese Embeddings
• LLM-Specific Utility for Retrieval-Augmented Generation

Related papers

• The Price of Isolation: Estimating the Ecosystem Cost of Symmetric Two-Sided A/B Testing
• SCAR: Semantic Continuity-Aware Retrieval for Efficient Context Expansion in RAG
• MixLoRA-DSI: Dynamically Expandable Mixture-of-LoRA Experts for Rehearsal-Free Generative Retrieval over Dynamic Corpora
• QProtFuse: Query-Guided Layer Fusion for Retrieval-Augmented Protein Function Prediction
• DualView: Adaptive Local-Global Fusion for Multi-Hop Document Reranking
• How Do LLM-Generated Texts Impact Term-Based Retrieval Models?