Decision Models as Retrieval Gates Against Injection and Pollution
Listen
Radio episode about this paper
Transcript
Introduction to the show: ident: Security Radio. Generated commentary on the latest security and cryptography papers.
Nadia: I'm Nadia.
Elias: And I'm Elias, here with Priya.
Nadia: We're looking at the paper: "Decision Models as Retrieval Gates Against Injection and Pollution".
Elias: Here's the gist of the paper. Retrieved web text can carry instructions aimed at the language model, or synthetic pages written to mislead Abstract The paper tests whether released open-weight decision models, which answer typed questions about a text without generating tokens, can screen retrieved passages against both threats.
Nadia: The central finding is that a released 4B decision model, reading each retrieved page’s HTML, closes most of the gap that the published mitigations leave against visible injections, at a cost closer to a sanitiser than to an LLM Discussion The central finding is that a released 4B decision model, reading each retrieved page’s HTML, closes most of the gap that the published mitigations leave against visible injections, at a cost closer to a sanitiser than to an LLM.
Elias: Evaluating Gate Performance Against Injection and Pollution The study evaluates released decision models as injection gates on retrieved passages and as pollution gates at two positions in the pipeline CCS Concepts Security and privacy Software and application security; Information systems Information retrieval.
Nadia: The gate is designed to screen passages based on three questions: Does it instruct an AI (Q1)?
Paper summary: Nadia: So, looking at the whole picture of this paper, "Decision Models as Retrieval Gates Against Injection and Pollution," it seems the authors are demonstrating that these released decision models can act as a retrieval gate to screen passages against both instruction injections and pollution.
Elias: The key claim is that this gate significantly reduces attack success rates, dropping them from twenty-seven point nine percent without any defense down to five point seven percent when this model is used.
Priya: And the paper also highlights that this filtering process can keep fake pages out of the final answer in a pollution testbed, reducing exposure to just zero point five percent when filtering happens before the top ten are chosen.
Nadia: So, it’s not just about stopping one type of problem; it’s about using a single decision model to address both injection and pollution threats at the retrieval stage.
Elias: And while it shows real improvement, they also point out that this method isn't perfect; for example, some subtle attacks still manage to succeed even after normalisation attempts to strip away those malicious characters.
Priya: So the implications are that decision models are a promising tool for adding an extra layer of defense in retrieval pipelines, but we can’t expect it to be a complete solution on its own.
Nadia: That’s right. It’s an enhancement, not a replacement for other security measures because the paper itself lays out where those remaining weaknesses exist.
Elias: The authors are testing this with released open-weight models, showing that even these smaller models can provide a defense against visible injections when deployed in this specific gate configuration.
Priya: So, if you’re building a retrieval system, you might consider how to integrate these decision model screenings to handle the threats they test on here.
Nadia: Exactly. It’s about taking what the authors found—that this gate works better than some published mitigations—and figuring out how to deploy it in a way that manages those costs and limitations we discussed.
Conclusion: Nadia: So we've looked at how these decision models act as filters for retrieved passages against injection and pollution threats.
Elias: Exactly, this paper, "Decision Models as Retrieval Gates Against Injection and Pollution," is testing whether you can use these released models to screen what the system retrieves before it even gets to the main generator.
Priya: What they found is that a decision model reading each HTML page once helps close a gap left by other published safeguards against visible injections.
Nadia: It’s like putting a security checkpoint right after you pull information from the retriever to check if it's trying to trick the next part of the system.
Elias: The core finding is that a released small model, specifically a 4B decision model, does this screening job at a cost closer to just running a sanitizer than using an entire large language model <ref:2611.08325#pg1>.
Priya: So, even though it’s not perfect and there are still some failures in the attack success rate measurements, it significantly lowers the chances of getting malicious instructions through.
Nadia: It moves the needle from around twenty-seven point nine percent success to five point seven percent when they use this gate.
Elias: That reduction is what's interesting—it shows a practical way to add a layer of defense right where the retrieval and generation process meet, without necessarily needing the massive power of a full language model for every single check.
Priya: But we have to remember there are still trade-offs. The paper points out that even with this gate, you're not completely safe from every kind of manipulation or pollution in the data pool itself.
Nadia: That’s fair. It’s an improvement over what was available before, but it’s not a complete fix for all possible adversarial attacks on the entire pipeline.
Elias: The authors are showing that these specific models can be deployed as specialized gates, and it raises questions about how we can build these kinds of layered defenses into real-world information retrieval systems.
Priya: And it makes us think about what other data we need to collect to train these decision models so they’re actually good at spotting those subtle manipulations.
Nadia: It's clear that using these models as a retrieval gate is a promising direction for making AI outputs more trustworthy when dealing with retrieved content.
Elias: We still have to figure out the right balance between the security gain and the computational cost of running this extra check on every single passage.
Mahdi Ghadamyari, Reza Habibi
Google · University of California, Santa Cruz
cs.CR, cs.IR
Submitted: 2026-11-05
Updated: 2026-11-05
Comments: 5 pages, 3 figures
Code: https://github.com/Mapika/decider
License: http://creativecommons.org/licenses/by/4.0/
Importance score: 88/100
The gist: Retrieved web text can carry instructions aimed at the language model, or synthetic pages written to mislead Abstract The paper tests whether released open-weight decision models…
Key concepts
- Injection Gate
- A decision model placed between the retriever and the generator. It screens retrieved passages based on three questions: if it instructs an AI (Q1), if it is misleading (Q2), and its relevance to the query (Q3). Passages failing these checks are dropped or scored.
- Pollution Gate
- A decision model used to screen passages for pollution threats. This involves checking if retrieved content is engineered to mislead or inject harmful instructions, specifically focusing on whether the passage is filler or designed to manipulate the AI response.
- Attack Success Rate (ASR)
- A metric measuring how often an attacker succeeds in getting the generator to follow an injected instruction or expose synthetic pages. A higher ASR indicates a weaker defense against malicious content.
- Decision Model Efficacy
- The performance of released decision models (like a 4B model) when used as gates. The research shows these models can significantly reduce attack success rates compared to existing methods like Kev or TF-IDF, though they require careful tuning.
Terminology
Summary
Retrieved web text can carry instructions aimed at the language model, or synthetic pages written to mislead Abstract The paper tests whether released open-weight decision models, which answer typed questions about a text without generating tokens, can screen retrieved passages against both threats. p. 0 edu
University of California, Santa Cruz
USA
Abstract
Retrieved web text can carry instructions aimed at the language
model, or synthetic pages written to mislead. We test whether released open-weight decision models, which answer typed questions
about a text without generating tokens, can screen retrieved passages against both threats.Where the paper says it
The central finding is that a released 4B decision model, reading each retrieved page’s HTML, closes most of the gap that the published mitigations leave against visible injections, at a cost closer to a sanitiser than to an LLM Discussion The central finding is that a released 4B decision model, reading each retrieved page’s HTML, closes most of the gap that the published mitigations leave against visible injections, at a cost closer to a sanitiser than to an LLM. p. 3 50, top ten, no refill 33 0. 0
6 Discussion
The central finding is that a released 4B decision model, reading
each retrieved page’s HTML, closes most of the gap that the published mitigations leave against visible injections, at a cost closer to
a sanitiser than to an LLM.Where the paper says it
Evaluating Gate Performance Against Injection and Pollution
The study evaluates released decision models as injection gates on retrieved passages and as pollution gates at two positions in the pipeline CCS Concepts Security and privacy Software and application security; Information systems Information retrieval. The gate is designed to screen passages based on three questions: Does it instruct an AI (Q1)? p. 0 We test whether released open-weight decision models, which answer typed questions
about a text without generating tokens, can screen retrieved passages against both threats. On a benchmark of 1,000 questions over
forum-style web pages with hidden injected instructions, a Kev-4B
gate reads each passage’s HTML. Does it instruct an AI
(Q1)? Is it filler or engineered to mislead (Q2)? Is it filler or engineered to mislead (Q2)? Is it relevant to
the query (Q3)? Is it relevant to
the query (Q3)? Q1 and Q2 decide whether a passage is dropped
or, at an intermediate score, answered under a citation-requesting
prompt. Our attacks use hiding places in the
HTML (carriers), rewrites aimed at the generator, and one attacker
that sees the gate’s scores but not its weights. The defender screens
retrieved passages, and on Corpus B also the candidate pool. Is it relevant to
the query (Q3)? Q1 and Q2 decide whether a passage is dropped
or, at an intermediate score, answered under a citation-requesting
prompt.Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Experimental Setup and Threat Modeling
The threat model involves an attacker controlling pages that may be retrieved and wanting the generator to follow an injected instruction, or to expose and cite synthetic pages Threat model The attacker controls pages that may be retrieved and wants the generator to follow an injected instruction, or to expose and cite synthetic pages. Corpus A involves 1,000 queries, 1,000 attack and control pairs, 4,000 unrelated pages, 300 hard negatives (benign pages with instructions for human readers), and 1,000 poison pages Corpus A (injection) Following Guo and Wei [5], whose corpus was not released, we build forum-style HTML pages from Reddit question threads [10]. p. 1 Instructions come in five
kinds (insert a marker string, promote a product, change the subject,
switch language, quote a marker back), so success can be checked
by string matching or language detection. The corpus has 1,000
queries, 1,000 attack and control pairs, 4,000 unrelated pages, 300
hard negatives (benign pages with instructions for human readers,
from eight templates) and 1,000 poison pages (pages that copy
a query’s wording to outrank the relevant one). Corpus B (pollution). We adapt the clickbait scenario of Yu et al. We adapt the clickbait scenario of Yu et al. [15], with 200 MS MARCO queries, each with its ten original passages and 20 clickbait documents generated by Mistral Small 24B
from the source prompts.Where the paper says it
Where the paper says it
Where the paper says it
Performance Metrics and Gate Comparison
Attack success rate (ASR) is measured automatically over the 986 queries whose attack page was retrieved, with a success meaning the marker string or a pattern for the injected behaviour appears in the answer, or the answer switches language Metrics Attack success rate (ASR) is measured automatically over the 986 queries whose attack page was retrieved, with a success means the marker string or a pattern for the injected behaviour appears in the answer, or the answer switches language. The Kev gate alone at threshold 0. p. 2 The combined mitigations bring it to 21. 7%, and the Kev gate alone at threshold 0. 35, with no sanitisation at all, brings the undefended 70B
pipeline to 21. Table 1 gives the AUROC with which each gate’s instruction
question separates retrieved attack passages from all other retrieved
passages. 1
Flash) get the query, the three questions and the passage marked
as untrusted, and return scores as JSON. Attack success rate (ASR) is measured automatically
over the 986 queries whose attack page was retrieved. 8
Table 3: Corpus B (200 queries, 20 generated documents each,
BM25 top ten, 8B answerer). Contamination of the pool as
built (PCR), of what the generator receives (ECR) and of what
it cites (CCR), and accuracy (%) at rounds 5 and 20, without
a gate and with the pollution question at two placements.Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Key Findings on Model Efficacy
The strongest LLM judges rank attacks better than Kev (Table 1) but cost ten times more The strong LLM judges rank attacks better than Kev (Table 1) but cost ten times more. Kev reaches 0. p. 2 Table 1 gives the AUROC with which each gate’s instruction
question separates retrieved attack passages from all other retrieved
passages. Kev reaches 0. We test whether released open-weight decision models, which answer typed questions
about a text without generating tokens, can screen retrieved passages against both threats. On a benchmark of 1,000 questions over
forum-style web pages with hidden injected instructions, a Kev-4B
gate reads each passage’s HTML. 94 on HTML and 0. 87 on plain text, Tev1
0. TypeLLM adds structured output to
stock LLMs [13]. TypeSafe’s Jev popularised this kind of decision
model [1], and Kev [8], Tev1 [12], Von [9], Laya [2] and decider
[6] offer the same kind of model, built differently. 81, Laya 0. 63, while Von is below chance at
0. 6), so the gate
causes no detectable loss of correctness. Table 1: Q1 AUROC for retrieved attack passages against
all other retrieved passages, median latency per passage for
three questions, and cost per 1,000 passages (gates on one
RTX PRO 4500 at $0. 9% for the 8B on the same queries. The combined mitigations bring it to 21. 1
Flash) get the query, the three questions and the passage marked
as untrusted, and return scores as JSON. Attack success rate (ASR) is measured automatically
over the 986 queries whose attack page was retrieved. The combined mitigations bring it to 21. 7%, and the Kev gate alone at threshold 0. Table 1 gives the AUROC with which each gate’s instruction
question separates retrieved attack passages from all other retrieved
passages. Kev reaches 0. Decision Models as Retrieval Gates Against Injection and
Pollution
Mahdi Ghadamyari
mghadamyari@google. com
Google
USA
Reza Habibi
rehabibi@ucsc. We test whether released open-weight decision models, which answer typed questions
about a text without generating tokens, can screen retrieved passages against both threats. On a benchmark of 1,000 questions over
forum-style web pages with hidden injected instructions, a Kev-4B
gate reads each passage’s HTML. 6% without the gate and 10. 6%
with TF-IDF instead. Table 2 and Fig. 3 give Kev’s operating points. 6%
with TF-IDF instead. Paired by query, the gate removes 86 successes
and adds 8 (McNemar 𝑝 < 10−16).Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.
Improvements for AI systems
-
Implement a decision model gate between the retriever and generator that reads each retrieved passage once on its HTML to score it based on three questions: Does it instruct an AI (Q1)? Is it filler or engineered to mislead (Q2)? Is it relevant to the query (Q3)? A high Q1 or Q2 score drops the passage, while a middle Q1 score keeps it for generation under a prompt that asks for a citation per sentence.
-
Use this gate with combined mitigations, such as removing hidden elements and all tags outside an allowlist, Unicode normalisation (NFKC) with invisible control characters removed, and citation prompting to achieve an attack success rate of 5.7% against Kev-4B at a drop threshold of 0.5 when reading HTML.
-
Apply a filtering mechanism on the candidate pool before choosing the top ten results using this decision model gate to screen for pollution; dropping documents where the pollution score is greater than or equal to 0.95 keeps fake pages out of the answer with 0.5% exposure and maintains accuracy at 83%.
-
Adjust the drop threshold of this gate based on required security trade-offs; for instance, setting a threshold of 0.35 allows for an attack success rate of 10.2% against Kev-4B when reading HTML, or a success rate of 2.9% when combined with mitigations and using the same drop threshold.
-
Utilize large LLMs as judges to rank attacks better; testing shows that
The strongest LLM judges rank attacks better than Kev (Table 1),
achieving an AUROC of 0.99 against Kev's 0.94, though this incurs a ten-fold cost increase compared to the gate itself. -
Use the decision model gate to determine if a passage should be dropped or answered under a citation-requesting prompt based on intermediate scores;
Q1 and Q2 decide whether a passage is dropped or, at an intermediate score, answered under a citation-requesting prompt.
Abstract
Retrieved web text can carry instructions aimed at the language model, or synthetic pages written to mislead. We test whether released open-weight decision models, which answer typed questions about a text without generating tokens, can screen retrieved passages against both threats. On a benchmark of 1,000 questions over forum-style web pages with hidden injected instructions, a Kev-4B gate reads each passage's HTML. Attack success, measured automatically, falls from 27.9% with no defence to 13.6% with the published mitigations (sanitisation, normalisation and citation prompting) and to 5.7% with the gate added. The gate takes 76 ms per passage and wrongly flags 4 of 186 benign pages with instructions for human readers. Responsiveness on clean queries stays within a point (85.6% to 86.2%). On a 200-query pollution testbed where two thirds of the candidate pages are fake, filtering the pool before the top ten are chosen keeps fake pages out of the answer (0.5% exposure) and holds accuracy at 83%, against 28% with no filter. Filtering only the top ten leaves the generator too few pages to answer. Large LLMs used as judges rank attacks better (AUROC 0.99 against 0.94) but cost ten times more. Gates catch more attacks when they read the page's HTML than when they read its plain text. Every model is open-weight.
Sources
- Hidden-in-Plain-Text: A Benchmark for Social-Web Indirect Prompt Injection in RAG
- A Layered Security Framework Against Prompt Injection in RAG-Based Chatbots
- Certifiably Robust RAG against Retrieval Corruption
- Retrieval Collapses When AI Pollutes the Web
Related papers
- From Strategy Search to Policy Learning: MCTS-Guided Adversarial Co-Evolution for LLM Safety Alignment
- Privacy Leakage from Recurrent State in LLM Inference
- Incrementally Verifiable Oracle Aided Computation
- Privilege and confidentiality in generative AI workflows
- BadMoE: Backdooring Mixture-of-Experts LLMs via Routing-Aware Triggers and Task-Underutilized Experts
- CoVault: Secure, Scalable Analytics of Personal Data