When More Documents Hurt RAG: Measuring and Reducing Vector Search Dilution with Domain-Scoped Retrieval

arXiv:2606.11350 · cs.CL, cs.IR · Submitted 2026-06-09 · Read on arXiv

Listen

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: I'm Tom.

Jane: And I'm Jane, here with Lu, Meng and our in-house large language model, Lalam.

Tom: Today's paper: "When More Documents Hurt RAG".

Jane: The gist The precision–faithfulness paradox arises when scaling retrieval-augmented generation to large, heterogeneous document collections,

Tom: First, who's behind it and why it matters.

Paper summary: Tom: So we're talking about this paper called "When More Documents Hurt RAG: Measuring and Reducing Vector Search Dilution with Domain-Scoped Retrieval". What it claims is that standard retrieval augmented generation gets worse when you use really big, mixed collections of documents.

Jane: That’s the core idea, Tom. It points out that dense similarity in vector search starts failing when the document collection gets too large and too varied, and top-k retrieval ends up bringing back chunks that look semantically similar but are actually wrong for the specific context you need to answer a question.

Lu: The paper formalizes this failure mode as vector search dilution, defining it by an equation: delta of q k star equals one minus P global of q over P scoped of q, which is the fraction of retrieval set m sub r m equals belonging to the target category k star when retrieval ranges over all documents C, and P scoped of q is that same fraction when you only search within categories C k star >

Meng: So, what's the practical problem? The abstract gives an example from a Wyoming Department of Transportation corpus where scaling from fifty-four to one thousand one hundred twenty-eight documents, which is eighty-eight thousand nine hundred seven chunks, cut accuracy from seventy-five percent down to below forty percent <ref:2606.11350#pg1,Wyoming Department of Transportation corpus where scaling from 54 to 1,128>.

Tom: Exactly. It shows that even with hybrid dense plus sparse retrieval methods in place, this dilution happens. They found similar patterns across six other Department of Transportation organizations too.

Jane: And to test this, they set up something called MASDR-RAG for their controlled experiment over two hundred expert validated queries and five different LLM backbones and two different index stacks <ref:2606.11350#pg1>.

Lalam: In that setup, domain scoping is the load bearing component of the system architecture >

Lu: They found that hard partitioning, which is one way to scope retrieval using metadata like Neo4j filters, isn't safe. But they found that training free soft scoping is safe when the structure of those scopes actually exists in real life >

Meng: That’s interesting because it means you don't need to manually curate all that metadata just to get better results; you can discover these relevant scopes without doing all the heavy lifting upfront >

Tom: But then they ran into something else, which they call a precision faithfulness paradox when using multi-agent orchestration, regardless of how you control the format or the prompt.

Jane: That paradox is what makes this paper so important because it tells us what to prioritize for enterprise scale RAG systems. They suggest that the main fix is simple: scope retrieval first, then just perform a single synthesis call whenever you can >

Lu: They show that in their evaluation, domain scoping using organizational metadata significantly improved the precision at ten from zero point seven seven up to zero point eight six across five different LLM backbones and six corpora >

Meng: But they also showed that when you use multi-agent orchestration under a Gemini production stack, the faithfulness score actually dropped from zero point six one down to zero point three five because the evidence retrieved contains duplicate passages or near duplicates >

Tom: So the main point here is that while using domain scoping helps with retrieval quality, orchestrating multiple agents to synthesize answers across those retrieved chunks creates a problem with accuracy and faithfulness >

Jane: It means for most organizations, especially those working with large, heterogeneous documents like government records or technical manuals, you should focus on getting that initial scope right before you start layering on complex multi-agent logic >

Lu: The paper generalizes these findings across different models and indexing implementations, showing the preference for domain scoping is robust regardless of the specific model or how you build your index >

Meng: From a production standpoint, this suggests that for single organization corpora with stable scopes, using something like R2-ROUTED with a single synthesis call gets the highest correctness and recall at half MASDR-RAG’s LLM call budget >

Tom: So the practical advice is to scope first, then stick to one synthesis call for now. It steers us away from complex multi-agent setups unless you have genuinely multi domain corpora that need it >

Jane: Exactly. The paper suggests that the load bearing decision for enterprise scale RAG should be domain scoping if you are dealing with a single organization's documents >

Lu: The generalization part is key because they tested this on a nine source public composite corpus and on HotpotQA distractors, showing the dilution effect isn't just a Wyoming Department of Transportation thing >

Meng: One limitation they state is that the routing taxonomy is manually crafted, and it assumes you have explicit organizational metadata already during the ingestion process >

Tom: And another limitation is that the open source evaluations are limited to models with seven to eight billion parameters, so they haven't tested how this holds up on much larger architectures yet >

Jane: So what we leave with is that domain scoping using organizational metadata seems like a load bearing decision for RAG systems dealing with large document collections, but you have to be careful about how you orchestrate the final synthesis step.

Conclusion: Tom: So we're wrapping up this look at "When More Documents Hurt RAG: Measuring and Reducing Vector Search Dilution with Domain-Scoped Retrieval".

Jane: Basically, it’s about how scaling up your document collection for a retrieval augmented generation system actually makes the search less accurate.

Lu: The authors are looking at how dense similarity in vector search loses its ability to discriminate when you have a lot of different documents mixed together.

Meng: They found that adding more documents, even in structured systems like those used by the Wyoming Department of Transportation, really hurts precision.

Lalam: It’s not just about finding similar stuff; it’s about finding the *right* kind of similar stuff for the specific context you need.

Tom: So what does this mean for a company building these systems? If you just throw more data in, your answer quality drops significantly.

Jane: The fix they propose is domain scoping. You restrict the search to only the documents relevant to that specific organizational area using metadata instead of searching everything at once.

Lu: They found that hard partitioning with metadata filters wasn't safe, but soft scoping, where you let the structure of those categories guide it without pre-curated labels, worked when you tested it against real-world structures.

Meng: That’s a relief for deployment because we don't have to manually tag every single document perfectly before we can use that scoping.

Lalam: And they showed that this domain scoping method significantly boosted precision at ten across five different large language models and six different test sets.

Tom: So, the main point is that before you start complex multi-agent stuff for synthesis, you need to lock down that initial retrieval step with a solid scope.

Jane: That’s right. The paper points to domain scoping as the load bearing decision for enterprise scale RAG systems when dealing with large document collections from a single source.

Lu: It suggests that for many real-world scenarios, focusing on getting that organizational boundary correct is more important than trying to get perfect synthesis across too many agents immediately.

Meng: So we should probably look at implementing some form of metadata filtering before we spend time optimizing the multi-agent orchestration layer.

Lalam: If you’re dealing with a single organization's documents, using a trained router like R2-ROUTED with just one synthesis call seems to give you the highest correctness and recall at half the cost of running all those agents.

Tom: It sounds like the future direction here is moving away from just more data toward smarter, scoped data access.

Jane: Exactly. And if you want to see how this applies beyond government DOTs, we’ll be looking at how these scope mechanisms generalize across different document types and organizational structures next.

Nabaraj Subedi, Ahmed Abdelaty, Shivanand Venkanna Sheshappanavar

Department of Electrical Engineering & Computer Science · Department of Civil, Architectural Engineering & Construction Management

cs.CL, cs.IR

Submitted: 2026-06-09

Updated: 2026-10-14

Comments: 19 pages, 5 figures, 20 tables. Preprint under review at ACL ARR October

Code: https://github.com/langchain-ai/langchain

License: http://creativecommons.org/licenses/by/4.0/

Importance score: 81/100

The gist: The gist The precision–faithfulness paradox arises when scaling retrieval-augmented generation to large, heterogeneous document collections, and domain scoping using organizational metadata is

Key concepts

Vector Search Dilution: This failure mode occurs when using dense similarity search in RAG systems with large document collections. As the collection grows, the similarity scores become less reliable, causing top-k retrieval to pull chunks that are semantically similar but contextually wrong for the specific query.
Domain Scoping: This technique restricts the document search space by filtering documents based on organizational metadata (like a Neo4j filter). By limiting the search to relevant domains, it drastically reduces noise and improves retrieval accuracy, which is identified as the primary fix for dilution.
Precision-Faithfulness Paradox: This paradox arises when using multi-agent orchestration in RAG. While scoping improves precision (accuracy of retrieval), complex multi-agent synthesis can degrade faithfulness (how accurately the final answer reflects the evidence) because combining evidence from many sources becomes difficult.

Terminology

RAG — 3× in this paper · explained in 3 episodes
Accuracy — 1× in this paper · explained in 1 episode

Summary

The gist The precision–faithfulness paradox arises when scaling retrieval-augmented generation to large, heterogeneous document collections, and domain scoping using organizational metadata is identified as the key fix for vector search dilution

How it works

Retrieval-augmented generation degrades when scaled to large, heterogeneous document collections where dense similarity loses its discriminative power and top-k retrieval increasingly returns semantically similar but contextually incorrect chunks, a failure mode referred to as vector search dilution Even when using hybrid dense+sparse retrieval, this dilution was observed in a deployed Wyoming Department of Transportation corpus where scaling from 54 to 1,128 documents reduced accuracy from 75% to below 40% The formal definition of dilution is given by the equation δ(q, k⋆) = 1 − Pglobal(q) / Pscoped(q), where Pglobal(q) is the fraction of the retrieval set Rm(q) belonging to the target category k⋆ when retrieval ranges over all of C, and Pscoped(q) is the same fraction when retrieval is restricted to Ck⋆ Categories with smaller chunk populations suffer the most severe dilution (Design δ=0.53; Specs δ=0.43), while high-density categories largely resist it

Architecture and Analysis

The paper introduces MASDR-RAG and HYBRIDROUTED to address this dilution, with domain scoping using organizational metadata being the primary driver of improved retrieval performance The architecture has three components: (1) Domain-scoped retrieval, where each agent restricts the search to documents matching a Neo4j metadata filter, reducing the effective search space by 85–98%, (2) Hybrid routing that runs a fast regex matcher first and falls back to a zero-shot classifier using an LLM, and (3) Multiagent orchestration that dispatches to nine domain agents via function calling The per-agent search-space reduction under document series scoping achieves a weighted average of 90.4% across the eight scopable categories

Evaluation and Paradox

The evaluation across five LLM backbones, six corpora, and two index stacks indicates that domain scoping using organizational metadata is the key fix, significantly improving P@10 from 0.77 to 0.86 (p < 0.05) In contrast, multi-agent orchestration produces configuration-dependent results, creating what is called the precision–faithfulness paradox, where under a Gemini production stack it reduces RAGAS faithfulness from 0.61 to 0.35 (p < 0.01) This degradation arises from the difficulty of synthesizing answers across multiple sources when retrieved evidence contains dense, near arXiv:2606.11350v1 [cs.CL] 9 Jun 2026 duplicate passages

Practical Recommendations

The practical recommendation is simple: scope first, then perform a single synthesis call, reserving full multi-agent orchestration for genuinely multi-domain corpora paired with native-toolcall backbones The findings suggest that the load-bearing decision for enterprisescale RAG is domain scoping For single organization corpora with stable scopes, R2-ROUTED (trained BGE-M3 router, single synthesis call) attains the highest correctness and Recall@10 at half MASDR-RAG’s LLM-call budget

Generalization and Robustness

Cross-domain replications on a 9-source public composite corpus and on HotpotQA-distractor indicate that the dilution effect and the HYBRIDROUTED-over-MASDR-RAG preference are not unique to WYDOT The mechanism transfers to all three DOT corpora when measured at the granularity each producer treats as topical, with document series suffices for Wyoming and Colorado while section is required for California The right scope axis is whichever organizational unit the corpus’s producer treats as topical

Limitations

The precision–faithfulness paradox is established using an LLM-as-judge framework, and standard RAGAS metrics can inadvertently penalize multi-agent responses, meaning the measured drop in MASDR-RAG’s faithfulness serves as an upper bound Furthermore, the routing taxonomy is manually crafted and assumes the availability of explicit organizational metadata during corpus ingestion The open-source evaluations are limited to 7–8B-parameter models, leaving ≥ 70B-parameter architectures untested Our primary finding is that the load-bearing decision is domain scoping

Reproducibility

Code, the 200-query WYDOT suite, all 6 opencorpus ingest/eval scripts, and SLURM submission scripts are released at (anonymized github URL) All models are public HuggingFace checkpoints; commercial backbones use OpenRouter Full hyperparameters, hardware, and seeds in App.

Improvements for AI systems

Domain Scoping via Metadata Filtering: Implement retrieval by restricting search to specific organizational scopes using document metadata (e.g., Neo4j filters based on document series). This action reduces the effective search space by 85–98% (Table 13) and mitigates vector search dilution where dense similarity loses power.
Hybrid Routing for Deterministic Dispatch: Use a hybrid routing mechanism combining a fast regex matcher with an LLM-based classifier (Hybrid Router) to dispatch queries to the correct domain agent. This approach avoids routing errors, such as the Case 2 failure mode where the monolithic search returns irrelevant context, leading to higher correctness (HYBRID-ROUTED.83).
Single Synthesis Call for Faithfulness Preservation: Reserve full multi-agent orchestration only for genuinely multi-domain corpora paired with native-toolcall backbones. For single or stable organizational corpora, use a single synthesis step to prevent context fragmentation, as the paper shows that multi-round orchestration triggers a faithfulness collapse (precision–faithfulness paradox).
Adaptive Scoping Strategy: Implement a mechanism to select the appropriate scope axis based on corpus characteristics. For example, for California data where section is required, scope should be applied at the section level, as this restores correlation, showing that the right scope axis is whichever organizational unit the corpus’s producer treats as topical.
Router Optimization: Utilize a BGE-M3 linear-probe router ("R2) instead of simple regex to improve routing accuracy. The paper shows that R2 lifts accuracy +28.4 points and weighted F1 +24.4 points over R0 and achieves the highest correctness on WYDOT 200-q (R2 attains the highest correctness (0.303)").

Abstract

Retrieval-augmented generation (RAG) degrades when applied to large, heterogeneous document collections. In RAG systems, dense similarity becomes less discriminative, causing top- k retrieval to return semantically similar but contextually incorrect chunks. We refer to this failure mode as vector search dilution. Even under hybrid dense+sparse retrieval, we first observed it in a deployed Wyoming Department of Transportation (WYDOT) system; in a controlled experiment, growing the corpus from 54 to 1,128 documents cut precision@10 by 63% and recall of the answer-bearing document by 23%. Similar patterns were observed in six other DOTs. To diagnose this issue, we introduce MASDR-RAG (Metadata-Scoped Domain Retrieval for RAG) as a controlled testbed over 200 human-validated queries, five LLM backbones, six public benchmark corpora, and two index stacks. In MASDR-RAG, domain scoping is the load-bearing component. We found hard partitioning to be unsafe, whereas training-free soft scoping is safe when the scope structure is real. Furthermore, we can discover these scopes without hand-curated metadata. However, multi-agent orchestration triggers a precision-faithfulness paradox that survives format controls, prompt controls, and blind human evaluation. Based on these varied outcomes, our practical recommendation is simple: soft-scope first, then perform a single synthesis call.