Indirect Prompt Injection in the Wild: An Empirical Study of Prevalence, Techniques, and Objectives

summary

The gist

As LLMs are increasingly integrated into systems that browse, retrieve, summarize, and act on web content…

In short

This study empirically analyzed indirect prompt injections found at scale across webpages and HTTP responses. It found that these attacks target machines over humans, are highly structured around specific templates, and are often deployed invisibly through channels like HTTP headers. Defenses should focus on detecting task overrides and targeting the most prevalent lexical templates.

Key concepts

Indirect Prompt Injection
This refers to malicious instructions hidden within a webpage or HTTP response that are not directly visible in the main content. Instead, they are designed to trick an AI model into following unintended commands by exploiting how the web content is processed.
Task Override
This is the most common prompting technique observed, where attackers attempt to directly replace or override the AI model's current instructions with a new command. Defenses should prioritize detecting this direct replacement of the existing task.
Invisibility Techniques
Attackers frequently hide their prompts using methods like manipulating color and contrast, blocking parts of the screen (occlusion), or hiding content within viewports. These techniques aim to make the malicious instructions undetectable by standard inspection tools.
Task-Specific Objectives
The paper categorizes injection goals into offensive uses like system disruption and defensive uses like data protection. Understanding these objectives helps tailor security measures to mitigate specific types of threats.

Terminology used across episodes

This episode discusses

The paper

Indirect Prompt Injection in the Wild: An Empirical Study of Prevalence, Techniques, and Objectives · Read on arXiv

Soheil Khodayari, Xuenan Zhang, Bhupendra Acharya, Giancarlo Pellegrino

CISPA Helmholtz Center for Information Security · University of Louisiana

Transcript

Introduction to the show: ident: Security Radio. Generated commentary on the latest security and cryptography papers.

Nadia: We're looking at the paper: "Indirect Prompt Injection in the Wild".

Elias: Here's the gist of the paper. As LLMs are increasingly integrated into systems that browse, retrieve, summarize, and act on web content, webpages have become an untrusted input vector for downstream model behavior <ref:2604.27202#pg2> This paper presents one of the first large-scale empirical analyses of indirect prompt injections in webpages and HTTP responses Pruning redundant reasoning tokens Indirect prompt injection is already present at scale on the web, with 15.3K confirmed instances on 11.7K pages across a large dataset <ref:2604.27202#pg9> The phenomenon is highly structured, as 54 prompt templates account for 95% of all cases <ref:2604.27202#pg10> These injections are shaped by multiple incentives, spanning roughly 1.5K reputation-manipulation prompts, 4K data-protection prompts, and 3K AI-bot-identification prompts A key finding is that most instructions target machines rather than humans, with about 70% appearing in non-rendered HTML How the training signal is built The study analyzes web pages and HTTP responses from three complementary sources, including Common Crawl, Shodan, and Censys A streaming pipeline was developed to process a subset of the October 2025 Common Crawl corpus, yielding approximately half of the corpus overall The research questions focus on prevalence, attacker objectives, deployment strategies, ecosystem distribution and persistence, and practical effectiveness across page representations Where the gains come from The analysis reveals six distinct objective categories spanning offensive uses like System Disruption or Degradation and defensive uses such as Data Protection Offensive objectives are dominated by Garbage Injection (Corruption), which accounts for 8,469 of the 8,894 instances Defensive objectives include Data Protection and AI Bot Identification, while Generic Content Override captures prompts whose override intent is clear but whose downstream purpose cannot be reliably determined The dominant prompting technique across all objective classes is Task Override, which is nearly universal (99% of injections) Pruning redundant reasoning tokens Inclusion techniques reveal that prompt injections are deployed through a small number of recurring channels, with nearly all instances appearing either in HTTP response headers or response bodies Invisibility is the dominant deployment property, as approximately 87% of injections are non-visible The most prevalent invisibility techniques include color and contrast manipulation (2,397 cases), occlusion (1,860), and viewport-based hiding (1,802) Pruning redundant reasoning tokens The effectiveness of in-page prompt injections depends more on how page content is exposed to the model than on model size alone The plain-text representation yields the highest attack-effectiveness rate (3.

Title and authors: Elias: Let’s talk about who did this work. It’s written by Soheil Khodayari from Germany, Xuenan Zhang from the CISPA Helmholtz Center for Information Security, and Bhupendra Acharya from the University of Louisiana >

Nadia: These are researchers focused on exploitation and defense in practice, so they have a real understanding of how these things actually work against systems >

Priya: I wonder what their focus is when they look at this whole phenomenon. Are they focusing more on the attack side or the defensive side initially?

Elias: They start by establishing the prevalence and structure of the attacks themselves, looking at deployment strategies and ecosystem distribution >

Nadia: They’re trying to map out where these injections are happening across the web and what goals attackers have when they deploy them >

The paper's summary: Priya: So, if we look at the summary of "Indirect Prompt Injection in the Wild: An Empirical Study of Prevalence, Techniques, and Objectives," what’s the big picture they’re trying to convey about what these attacks are actually doing?

Nadia: They found that most of these instructions are targeting machines rather than actual humans, with about seventy percent appearing in non-rendered HTML >

Elias: That machine focus is key. And beneath all that variety, the paper says ninety-nine percent of injections attempt direct task override, often using jailbreak language to reinforce that intent >

Priya: Task override versus other things. Does that mean we should be looking for something very specific in a prompt when we try to defend against it?

Nadia: Yes, they suggest defenses should focus on detecting the direct replacement of the model’s current task instead of just trying to catch every subtle persuasion attempt >

Elias: That makes sense. It shifts our focus from vague language to identifying that core instruction that’s trying to hijack what the AI is supposed to be doing >

The paper's improvements: Nadia: The authors actually suggest some things they think we should do next based on their findings. They point out that prompt injections are deployed through a small number of recurring channels, mostly in HTTP response headers or bodies >

Elias: And one of the main things they highlight is invisibility; about eighty-seven percent of these injections are non-visible to the end user >

Priya: So, if they’re invisible most of the time, what does that imply for how we design detection systems? Do we just need to look at what's rendered on screen?

Nadia: They suggest prioritizing non-visible channels like HTTP headers and structured data objects for signals because those are treated as non-visible by construction >

Elias: That ties into the idea that effectiveness depends more on how page content is exposed to the model than just how big the model is, especially when structural cues are preserved >

Conclusion: Elias: To wrap up this paper, we see that indirect prompt injection is a huge part of how websites are responding to AI agents now, and it’s not just some isolated security issue >

Nadia: It’s a widespread pattern, driven by these structured templates and a massive focus on task override >

Priya: From my side, what I take away is that the real challenge isn't just stopping one type of injection; it’s dealing with this whole ecosystem where these prompts are used for disruption and degradation >

Elias: Agreed. And the paper makes it clear that defenses need to be looking at those recurring channels and those specific task override patterns we discussed >

Nadia: So, when you look at "Indirect Prompt Injection in the Wild: An Empirical Study of Prevalence, Techniques, and Objectives," remember that they’ve mapped out the structure of these attacks across a huge amount of web traffic >

More episodes

← Home