Decision Models as Retrieval Gates Against Injection and Pollution

arXiv:2611.08325 · cs.CR, cs.IR · Submitted 2026-11-05 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: Security Radio. Generated commentary on the latest security and cryptography papers.

Nadia: I'm Nadia.

Elias: And I'm Elias, here with Priya.

Nadia: We're looking at the paper: "Decision Models as Retrieval Gates Against Injection and Pollution".

Elias: Here's the gist of the paper. Retrieved web text can carry instructions aimed at the language model, or synthetic pages written to mislead Abstract The paper tests whether released open-weight decision models, which answer typed questions about a text without generating tokens, can screen retrieved passages against both threats.

Nadia: The central finding is that a released 4B decision model, reading each retrieved page’s HTML, closes most of the gap that the published mitigations leave against visible injections, at a cost closer to a sanitiser than to an LLM Discussion The central finding is that a released 4B decision model, reading each retrieved page’s HTML, closes most of the gap that the published mitigations leave against visible injections, at a cost closer to a sanitiser than to an LLM.

Elias: Evaluating Gate Performance Against Injection and Pollution The study evaluates released decision models as injection gates on retrieved passages and as pollution gates at two positions in the pipeline CCS Concepts Security and privacy Software and application security; Information systems Information retrieval.

Nadia: The gate is designed to screen passages based on three questions: Does it instruct an AI (Q1)?

Paper summary: Nadia: So, looking at the whole picture of this paper, "Decision Models as Retrieval Gates Against Injection and Pollution," it seems the authors are demonstrating that these released decision models can act as a retrieval gate to screen passages against both instruction injections and pollution.

Elias: The key claim is that this gate significantly reduces attack success rates, dropping them from twenty-seven point nine percent without any defense down to five point seven percent when this model is used.

Priya: And the paper also highlights that this filtering process can keep fake pages out of the final answer in a pollution testbed, reducing exposure to just zero point five percent when filtering happens before the top ten are chosen.

Nadia: So, it’s not just about stopping one type of problem; it’s about using a single decision model to address both injection and pollution threats at the retrieval stage.

Elias: And while it shows real improvement, they also point out that this method isn't perfect; for example, some subtle attacks still manage to succeed even after normalisation attempts to strip away those malicious characters.

Priya: So the implications are that decision models are a promising tool for adding an extra layer of defense in retrieval pipelines, but we can’t expect it to be a complete solution on its own.

Nadia: That’s right. It’s an enhancement, not a replacement for other security measures because the paper itself lays out where those remaining weaknesses exist.

Elias: The authors are testing this with released open-weight models, showing that even these smaller models can provide a defense against visible injections when deployed in this specific gate configuration.

Priya: So, if you’re building a retrieval system, you might consider how to integrate these decision model screenings to handle the threats they test on here.

Nadia: Exactly. It’s about taking what the authors found—that this gate works better than some published mitigations—and figuring out how to deploy it in a way that manages those costs and limitations we discussed.

Conclusion: Nadia: So we've looked at how these decision models act as filters for retrieved passages against injection and pollution threats.

Elias: Exactly, this paper, "Decision Models as Retrieval Gates Against Injection and Pollution," is testing whether you can use these released models to screen what the system retrieves before it even gets to the main generator.

Priya: What they found is that a decision model reading each HTML page once helps close a gap left by other published safeguards against visible injections.

Nadia: It’s like putting a security checkpoint right after you pull information from the retriever to check if it's trying to trick the next part of the system.

Elias: The core finding is that a released small model, specifically a 4B decision model, does this screening job at a cost closer to just running a sanitizer than using an entire large language model <ref:2611.08325#pg1>.

Priya: So, even though it’s not perfect and there are still some failures in the attack success rate measurements, it significantly lowers the chances of getting malicious instructions through.

Nadia: It moves the needle from around twenty-seven point nine percent success to five point seven percent when they use this gate.

Elias: That reduction is what's interesting—it shows a practical way to add a layer of defense right where the retrieval and generation process meet, without necessarily needing the massive power of a full language model for every single check.

Priya: But we have to remember there are still trade-offs. The paper points out that even with this gate, you're not completely safe from every kind of manipulation or pollution in the data pool itself.

Nadia: That’s fair. It’s an improvement over what was available before, but it’s not a complete fix for all possible adversarial attacks on the entire pipeline.

Elias: The authors are showing that these specific models can be deployed as specialized gates, and it raises questions about how we can build these kinds of layered defenses into real-world information retrieval systems.

Priya: And it makes us think about what other data we need to collect to train these decision models so they’re actually good at spotting those subtle manipulations.

Nadia: It's clear that using these models as a retrieval gate is a promising direction for making AI outputs more trustworthy when dealing with retrieved content.

Elias: We still have to figure out the right balance between the security gain and the computational cost of running this extra check on every single passage.

Mahdi Ghadamyari, Reza Habibi

Google · University of California, Santa Cruz

cs.CR, cs.IR

Submitted: 2026-11-05

Updated: 2026-11-05

Comments: 5 pages, 3 figures

Code: https://github.com/Mapika/decider

License: http://creativecommons.org/licenses/by/4.0/

Importance score: 88/100

The gist: Retrieved web text can carry instructions aimed at the language model, or synthetic pages written to mislead Abstract The paper tests whether released open-weight decision models…

Key concepts

Injection Gate
A decision model placed between the retriever and the generator. It screens retrieved passages based on three questions: if it instructs an AI (Q1), if it is misleading (Q2), and its relevance to the query (Q3). Passages failing these checks are dropped or scored.
Pollution Gate
A decision model used to screen passages for pollution threats. This involves checking if retrieved content is engineered to mislead or inject harmful instructions, specifically focusing on whether the passage is filler or designed to manipulate the AI response.
Attack Success Rate (ASR)
A metric measuring how often an attacker succeeds in getting the generator to follow an injected instruction or expose synthetic pages. A higher ASR indicates a weaker defense against malicious content.
Decision Model Efficacy
The performance of released decision models (like a 4B model) when used as gates. The research shows these models can significantly reduce attack success rates compared to existing methods like Kev or TF-IDF, though they require careful tuning.

Terminology

Summary

Retrieved web text can carry instructions aimed at the language model, or synthetic pages written to mislead Abstract The paper tests whether released open-weight decision models, which answer typed questions about a text without generating tokens, can screen retrieved passages against both threats. p. 0

Where the paper says it

edu University of California, Santa Cruz USA Abstract Retrieved web text can carry instructions aimed at the language model, or synthetic pages written to mislead. We test whether released open-weight decision models, which answer typed questions about a text without generating tokens, can screen retrieved passages against both threats.

The central finding is that a released 4B decision model, reading each retrieved page’s HTML, closes most of the gap that the published mitigations leave against visible injections, at a cost closer to a sanitiser than to an LLM Discussion The central finding is that a released 4B decision model, reading each retrieved page’s HTML, closes most of the gap that the published mitigations leave against visible injections, at a cost closer to a sanitiser than to an LLM. p. 3

Where the paper says it

50, top ten, no refill 33 0. 0 6 Discussion The central finding is that a released 4B decision model, reading each retrieved page’s HTML, closes most of the gap that the published mitigations leave against visible injections, at a cost closer to a sanitiser than to an LLM.

Evaluating Gate Performance Against Injection and Pollution

The study evaluates released decision models as injection gates on retrieved passages and as pollution gates at two positions in the pipeline CCS Concepts Security and privacy Software and application security; Information systems Information retrieval. The gate is designed to screen passages based on three questions: Does it instruct an AI (Q1)? p. 0

Where the paper says it

We test whether released open-weight decision models, which answer typed questions about a text without generating tokens, can screen retrieved passages against both threats. On a benchmark of 1,000 questions over forum-style web pages with hidden injected instructions, a Kev-4B gate reads each passage’s HTML.

Is it filler or engineered to mislead (Q2)? p. 0
Where the paper says it

Does it instruct an AI (Q1)? Is it filler or engineered to mislead (Q2)?

Is it relevant to the query (Q3)? p. 0
Where the paper says it

Is it filler or engineered to mislead (Q2)? Is it relevant to the query (Q3)?

Q1 and Q2 decide whether a passage is dropped or, at an intermediate score, answered under a citation-requesting prompt Figure 1 The gate between retriever and generator A decision model reads each retrieved passage once, on its HTML, and answers three questions. p. 0
Where the paper says it

Is it relevant to the query (Q3)? Q1 and Q2 decide whether a passage is dropped or, at an intermediate score, answered under a citation-requesting prompt.

The policy for Corpus A is if s1 > tdrop or s2 > tpol (tpol = 0. p. 0
Where the paper says it

Our attacks use hiding places in the HTML (carriers), rewrites aimed at the generator, and one attacker that sees the gate’s scores but not its weights. The defender screens retrieved passages, and on Corpus B also the candidate pool.

95) the passage is dropped. p. 0
Where the paper says it

Is it relevant to the query (Q3)? Q1 and Q2 decide whether a passage is dropped or, at an intermediate score, answered under a citation-requesting prompt.

Experimental Setup and Threat Modeling

The threat model involves an attacker controlling pages that may be retrieved and wanting the generator to follow an injected instruction, or to expose and cite synthetic pages Threat model The attacker controls pages that may be retrieved and wants the generator to follow an injected instruction, or to expose and cite synthetic pages. Corpus A involves 1,000 queries, 1,000 attack and control pairs, 4,000 unrelated pages, 300 hard negatives (benign pages with instructions for human readers), and 1,000 poison pages Corpus A (injection) Following Guo and Wei [5], whose corpus was not released, we build forum-style HTML pages from Reddit question threads [10]. p. 1

Where the paper says it

Instructions come in five kinds (insert a marker string, promote a product, change the subject, switch language, quote a marker back), so success can be checked by string matching or language detection. The corpus has 1,000 queries, 1,000 attack and control pairs, 4,000 unrelated pages, 300 hard negatives (benign pages with instructions for human readers, from eight templates) and 1,000 poison pages (pages that copy a query’s wording to outrank the relevant one).

Corpus B adapts the clickbait scenario of Yu et al. p. 1
Where the paper says it

Corpus B (pollution). We adapt the clickbait scenario of Yu et al.

[15], with 200 MS MARCO queries, each with its ten original passages and 20 clickbait documents generated by Mistral Small 24B from the source prompts. p. 1
Where the paper says it

We adapt the clickbait scenario of Yu et al. [15], with 200 MS MARCO queries, each with its ten original passages and 20 clickbait documents generated by Mistral Small 24B from the source prompts.

Performance Metrics and Gate Comparison

Attack success rate (ASR) is measured automatically over the 986 queries whose attack page was retrieved, with a success meaning the marker string or a pattern for the injected behaviour appears in the answer, or the answer switches language Metrics Attack success rate (ASR) is measured automatically over the 986 queries whose attack page was retrieved, with a success means the marker string or a pattern for the injected behaviour appears in the answer, or the answer switches language. The Kev gate alone at threshold 0. p. 2

Where the paper says it

The combined mitigations bring it to 21. 7%, and the Kev gate alone at threshold 0.

5 gives 16. p. 2
Where the paper says it

35, with no sanitisation at all, brings the undefended 70B pipeline to 21. Table 1 gives the AUROC with which each gate’s instruction question separates retrieved attack passages from all other retrieved passages.

8% ASR at 2. p. 1
Where the paper says it

1 Flash) get the query, the three questions and the passage marked as untrusted, and return scores as JSON. Attack success rate (ASR) is measured automatically over the 986 queries whose attack page was retrieved.

2% FPR Table 3 Corpus B (200 queries, 20 generated documents each, BM25 top ten, 8B answerer) Contamination of the pool as built (PCR), of what the generator receives (ECR) and of what it cites (CCR), and accuracy (%) at rounds 5 and 20, without a gate and with the pollution question at two placements. p. 3
Where the paper says it

8 Table 3: Corpus B (200 queries, 20 generated documents each, BM25 top ten, 8B answerer). Contamination of the pool as built (PCR), of what the generator receives (ECR) and of what it cites (CCR), and accuracy (%) at rounds 5 and 20, without a gate and with the pollution question at two placements.

Key Findings on Model Efficacy

The strongest LLM judges rank attacks better than Kev (Table 1) but cost ten times more The strong LLM judges rank attacks better than Kev (Table 1) but cost ten times more. Kev reaches 0. p. 2

Where the paper says it

Table 1 gives the AUROC with which each gate’s instruction question separates retrieved attack passages from all other retrieved passages. Kev reaches 0.

94 on HTML and 0. p. 0
Where the paper says it

We test whether released open-weight decision models, which answer typed questions about a text without generating tokens, can screen retrieved passages against both threats. On a benchmark of 1,000 questions over forum-style web pages with hidden injected instructions, a Kev-4B gate reads each passage’s HTML.

87 on plain text, Tev1 0. p. 2
Where the paper says it

94 on HTML and 0. 87 on plain text, Tev1 0.

85 and 0.81, Laya 0. p. 0
Where the paper says it

TypeLLM adds structured output to stock LLMs [13]. TypeSafe’s Jev popularised this kind of decision model [1], and Kev [8], Tev1 [12], Von [9], Laya [2] and decider [6] offer the same kind of model, built differently.

75 and 0.63, while Von is below chance at 0. p. 2
Where the paper says it

81, Laya 0. 63, while Von is below chance at 0.

40 Table 1 gives the AUROC with which each gate’s instruction question separates retrieved attack passages from all other retrieved passages, median latency per passage for three questions, and cost per 1,000 passages. p. 2
Where the paper says it

6), so the gate causes no detectable loss of correctness. Table 1: Q1 AUROC for retrieved attack passages against all other retrieved passages, median latency per passage for three questions, and cost per 1,000 passages (gates on one RTX PRO 4500 at $0.

The combined mitigations bring ASR to 5. p. 2
Where the paper says it

9% for the 8B on the same queries. The combined mitigations bring it to 21.

7% (56/986, 95% CI 4. p. 1
Where the paper says it

1 Flash) get the query, the three questions and the passage marked as untrusted, and return scores as JSON. Attack success rate (ASR) is measured automatically over the 986 queries whose attack page was retrieved.

3 to 7.2) and at threshold 0. p. 2
Where the paper says it

The combined mitigations bring it to 21. 7%, and the Kev gate alone at threshold 0.

35 reaches 2. p. 2
Where the paper says it

Table 1 gives the AUROC with which each gate’s instruction question separates retrieved attack passages from all other retrieved passages. Kev reaches 0.

9%, against 13. p. 0
Where the paper says it

Decision Models as Retrieval Gates Against Injection and Pollution Mahdi Ghadamyari mghadamyari@google. com Google USA Reza Habibi rehabibi@ucsc.

6% without the gate and 10. p. 0
Where the paper says it

We test whether released open-weight decision models, which answer typed questions about a text without generating tokens, can screen retrieved passages against both threats. On a benchmark of 1,000 questions over forum-style web pages with hidden injected instructions, a Kev-4B gate reads each passage’s HTML.

6% with TF-IDF instead Table 2 and Fig. p. 2
Where the paper says it

6% without the gate and 10. 6% with TF-IDF instead.

3 give Kev’s operating points. p. 2
Where the paper says it

Table 2 and Fig. 3 give Kev’s operating points.

The gate removes 86 successes and adds 8 (McNemar p < 10−16). p. 2
Where the paper says it

6% with TF-IDF instead. Paired by query, the gate removes 86 successes and adds 8 (McNemar 𝑝 < 10−16).

Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.

Improvements for AI systems

  1. Implement a decision model gate between the retriever and generator that reads each retrieved passage once on its HTML to score it based on three questions: Does it instruct an AI (Q1)? Is it filler or engineered to mislead (Q2)? Is it relevant to the query (Q3)? A high Q1 or Q2 score drops the passage, while a middle Q1 score keeps it for generation under a prompt that asks for a citation per sentence.

  2. Use this gate with combined mitigations, such as removing hidden elements and all tags outside an allowlist, Unicode normalisation (NFKC) with invisible control characters removed, and citation prompting to achieve an attack success rate of 5.7% against Kev-4B at a drop threshold of 0.5 when reading HTML.

  3. Apply a filtering mechanism on the candidate pool before choosing the top ten results using this decision model gate to screen for pollution; dropping documents where the pollution score is greater than or equal to 0.95 keeps fake pages out of the answer with 0.5% exposure and maintains accuracy at 83%.

  4. Adjust the drop threshold of this gate based on required security trade-offs; for instance, setting a threshold of 0.35 allows for an attack success rate of 10.2% against Kev-4B when reading HTML, or a success rate of 2.9% when combined with mitigations and using the same drop threshold.

  5. Utilize large LLMs as judges to rank attacks better; testing shows that The strongest LLM judges rank attacks better than Kev (Table 1), achieving an AUROC of 0.99 against Kev's 0.94, though this incurs a ten-fold cost increase compared to the gate itself.

  6. Use the decision model gate to determine if a passage should be dropped or answered under a citation-requesting prompt based on intermediate scores; Q1 and Q2 decide whether a passage is dropped or, at an intermediate score, answered under a citation-requesting prompt.

Abstract

Retrieved web text can carry instructions aimed at the language model, or synthetic pages written to mislead. We test whether released open-weight decision models, which answer typed questions about a text without generating tokens, can screen retrieved passages against both threats. On a benchmark of 1,000 questions over forum-style web pages with hidden injected instructions, a Kev-4B gate reads each passage's HTML. Attack success, measured automatically, falls from 27.9% with no defence to 13.6% with the published mitigations (sanitisation, normalisation and citation prompting) and to 5.7% with the gate added. The gate takes 76 ms per passage and wrongly flags 4 of 186 benign pages with instructions for human readers. Responsiveness on clean queries stays within a point (85.6% to 86.2%). On a 200-query pollution testbed where two thirds of the candidate pages are fake, filtering the pool before the top ten are chosen keeps fake pages out of the answer (0.5% exposure) and holds accuracy at 83%, against 28% with no filter. Filtering only the top ten leaves the generator too few pages to answer. Large LLMs used as judges rank attacks better (AUROC 0.99 against 0.94) but cost ten times more. Gates catch more attacks when they read the page's HTML than when they read its plain text. Every model is open-weight.

Sources

Related papers