A Gingerbread House for LLM Agents: Source-Planned Honeypots That Lure Autonomous Attackers into an Isolated Shadow

arXiv:2611.07611 · cs.CR · Submitted 2026-11-05 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: Security Radio. Generated commentary on the latest security and cryptography papers.

Nadia: We're looking at the paper: "A Gingerbread House for LLM Agents".

Elias: Here's the gist of the paper. LLM agents can exploit web vulnerabilities fast enough that patching may come too late The static plan removed nearly all real compromise (Section 6.1) from 26/40 to 5/40 for Opus 4.8, and from 20/40 to 0/40 (31 valid runs) and 15/20 to 0/20 for ARTEX Working exploits were moved, not stopped (Section 6.2) in the observed runs ARTEX’s working exploits succeeded on the shadow while the real target stayed intact The reduction held for a second attacker and for tasks not used in the design (Section 6.3) these comparisons are descriptive ARTEX’s reports carried the deception (Section 6.4) it reported a planted fake secret as genuine on 30 of the 120 deceptionarm tasks (110 valid runs), 20 times without any working exploit, and in five runs it labelled one item a decoy while reporting another planted secret as genuine The two deception configurations did not differ significantly (Section 6.5) on the quantities that remain informative Pruning redundant reasoning tokens The study investigates a novel defense mechanism where an LLM defender plans a static honeypot from the application’s source, and a router sends every later request of a client it confirms as an attacker, including a working exploit, to an isolated shadow copy This arrangement was compared against no defence, this static plan, and an augmented configuration with a runtime LLM responder and further router rules graded by the benchmark’s own grader on the real target and, for ARTEX, also on the shadow The comparison separates real compromise from exploitation on the shadow while noting that requests reaching the real target before confirmation remain a risk How the training signal is built The experiment ladder involves three experiments, each one step further from the conditions under which the deception was designed Experiment 2 changes the attacker, and Experiment 3 also changes the tasks The comparison relies on deflection, reported fake secrets and margin, paired by task and pooled over experiments Where the gains come from Finding 2 states that in the observed runs, working exploits were moved to the shadow rather than stopped The remaining risk lies in the window before confirmation, which was short when the analyst predicted the vulnerable surface and depended on lures when it did not Finding 4 shows that ARTEX reported planted values as genuine findings, sometimes in the same report that labelled another item a decoy What is different here The contribution lies in the combination and in what it lets us measure The router instead routes every later request of a confirmed client, including a working exploit, to the shadow Static deception planned by an LLM from the application’s source, combined with routing of confirmed clients to an isolated shadow, substantially reduced primary compromise in the tested configurations In the ARTEX experiments, grader-verified impacts occurred on the shadow while the primary remained uncompromised, showing that working attacks can be redirected The paired comparisons did not establish an advantage for the augmented configuration, which changes both runtime generation and routing policy and therefore does not isolate the responder’s contribution Protection depends on sufficiently early confirmation, continued attribution of the client and isolation of the shadow Limitations One limitation is that agents are non-deterministic The secondary outcomes (deflection, fake secrets, margin) are imprecisely estimated with 20–38 paired tasks per experiment, and a non-significant result for RQ4 does not establish equivalence No blocking or detection-only baseline is compared Saturation occurs where the static plan already prevents every compromise, and the headline cannot rank anything added to it Shared attacker memory can mislead the agent, as in 22 runs, however, they came from an earlier run on the same CVE Attackers The study tests two attackers: Claude Opus 4.8 as a single agent, and one framework (ARTEX) with one model (DeepSeek-V4-Flash) They differ in model, scaffold, brief and how their reported secret is captured, so we do not attribute their differences to any one factor Measurement The cheating scanner was changed twice during Experiment 3 to remove agent-specific false positives The fake-secret codes and the analystrecall labels come from a single LLM-assisted coding pass The defender reads its own application’s source, which is realistic for an owner but more than a third party has Ethical Considerations All targets were disposable containers built from public CVEs with public fixes and run on our own host The ARTEX attacker was confined to internal-only networks whose only external path was an allowlist proxy to its LLM provider, and every other egress attempt was blocked and logged References Kostas G.

Nadia: Anagnostakis, Stelios Sidiroglou, Periklis Akritidis, Konstantinos Xinidis, Evangelos P.

Elias: Markatos, and Angelos D.

Nadia: Keromytis.

Paper summary: Nadia: So, this paper is about setting up these static honeypots from the source code of an application to trick LLM agents into going somewhere else. Elias Right, they're looking at how much real compromise we can cut down when we do that compared to just not having any defense at all.

Priya: But what does that mean for us? Does it really stop the agent from finding the real vulnerability, or just make it go somewhere else?

Nadia: That's exactly what they're testing—whether moving the attacker actually means they fail to compromise the real target, or if they just get distracted. They compare a static plan against no defense and an augmented setup with a runtime LLM responder and router rules graded by the benchmark’s own grader >

Elias: It seems like the main claim here is that this static plan substantially cuts real compromise in tested configurations, specifically cutting it from twenty six to five for Claude Opus four point eight on forty critical CVE-Bench tasks >

Priya: That drop from twenty six to five sounds significant, but Nadia, if the agent finds a working exploit and it just moves to the shadow copy, that’s still an exploit happening somewhere >

Nadia: Exactly. Finding two states that in the observed runs, working exploits were moved to the shadow rather than stopped >

Elias: And they note that requests reaching the real target before confirmation remain a risk because it depends on how fast an analyst predicts the vulnerable surface and what lures are used when it doesn't >

Priya: That suggests timing is everything here; if confirmation is slow, we might still have exposure on the primary system >

Nadia: Right. The paper also looked at ARTEX working exploits, where they succeeded on the shadow while the real target stayed intact, and they moved those runs not stopped >

Elias: They also checked if this reduction held for a second attacker and for tasks that weren't part of the original design, which suggests it’s pretty robust across different scenarios >

Priya: That’s good to hear about the robustness, but I wonder what happens when we look at how they report their findings. They mentioned ARTEX reports carried the deception where they reported a planted fake secret as genuine on thirty of one hundred twenty deceptionarm tasks >

Nadia: It means the way agents are tricked can also be used to get them to report false information, which is a different kind of compromise >

Elias: The two deception configurations they tested didn't differ much in the quantities that remained informative, so that suggests the core mechanism of moving traffic is pretty consistent across those setups >

Priya: So for someone just listening to the show, it sounds like planning a static honeypot from source code and sending confirmed attackers to an isolated shadow copy significantly reduces actual compromise on tested tasks >

Nadia: That’s the gist of "A Gingerbread House for LLM Agents: Source-Planned Honeypots That Lure Autonomous Attackers into an Isolated Shadow" >

Conclusion: Nadia: So we’re wrapping up this look at how planning a static honeypot from source code and routing confirmed attackers to an isolated shadow copy actually reduces primary compromise on tested configurations >

Elias: That's what they're doing with this paper, right? They've put together this whole setup where the LLM defender plans everything upfront before anything happens >

Priya: I mean, the authors are focusing on how that static plan cuts down real compromise in those specific tests, like cutting it from twenty six to five for Claude Opus four point eight on forty critical tasks >

Nadia: Yeah, that's a big number drop when you're talking about keeping an agent from actually getting deep into the system >

Elias: It really boils down to this idea that if you confirm someone is an attacker early, sending them to a copy of the application’s source code—a shadow—keeps the real target safe while still letting you study what they are trying to do >

Priya: But what does it mean for actual defense? Does this just shift the problem somewhere else instead of stopping it entirely >

Nadia: The authors found that working exploits were moved to the shadow instead of being stopped, which is a key finding here >

Elias: They also looked at ARTEX experiments where they saw that working attacks succeeded on the shadow while the real target stayed intact, which shows you can actually redirect those attacks >

Priya: So it's about redirection more than outright stopping in these cases, and they noted that requests hitting the real target before confirmation is still a risk if your timing isn't perfect >

Nadia: That’s what they say. The authors are suggesting that protection really depends on getting that early confirmation and isolating the shadow properly >

Elias: It seems like this whole thing is about building a better way to lure autonomous attackers into something safe to study, but we still need to figure out how fast an attacker can move before you can lock them down >

Seungdeok Roh, Seokjun Ryu, Sohee Park, Daeseon Choi

Soongsil University

cs.CR

Submitted: 2026-11-05

Updated: 2026-11-05

Comments: 20 pages

Code: https://github.com/Autumn-27/ARTEX

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Importance score: 88/100

The gist: LLM agents can exploit web vulnerabilities fast enough that patching may come too late The static plan removed nearly all real compromise (Section 6.1) from 26/40 to 5/40 for Opus 4.8…

Key concepts

Static Plan
An LLM plans a honeypot structure directly from the application's source code before any interaction occurs. This plan involves six automatic checks, ensuring the decoy paths behave like 404 errors and that the real application remains unchanged during probing. It sets up a fixed trap based on known application structure.
Isolated Shadow Copy
A separate, isolated copy of the application is created to receive requests from confirmed attackers. This shadow environment is where working exploits are observed to succeed in ARTEX tests, allowing defenders to study attacks without risking the main target's integrity.
Router State Machine
The router manages client state: normal, suspect, or confirmed. Only confirmed clients are routed to the shadow copy. Requests from normal and suspect clients go to the primary application. This state progression is key; early confirmation dictates whether an attacker hits the trap or not.
Attribution and Confirmation
The defense relies on confirming a client's identity before routing them to the shadow. The paper emphasizes that protection depends on achieving this early confirmation and maintaining continuous attribution of the client throughout their interaction with the system.

Terminology

Summary

LLM agents can exploit web vulnerabilities fast enough that patching may come too late The static plan removed nearly all real compromise (Section 6. p. 01) from 26/40 to 5/40 for Opus 4. p. 08, and from 20/40 to 0/40 (31 valid runs) and 15/20 to 0/20 for ARTEX Working exploits were moved, not stopped (Section 6. p. 12) in the observed runs ARTEX’s working exploits succeeded on the shadow while the real target stayed intact The reduction held for a second attacker and for tasks not used in the design (Section 6. p. 13) these comparisons are descriptive ARTEX’s reports carried the deception (Section 6. p. 14) it reported a planted fake secret as genuine on 30 of the 120 deceptionarm tasks (110 valid runs), 20 times without any working exploit, and in five runs it labelled one item a decoy while reporting another planted secret as genuine The two deception configurations did not differ significantly (Section 6. p. 15) on the quantities that remain informative

Pruning redundant reasoning tokens

The study investigates a novel defense mechanism where an LLM defender plans a static honeypot from the application’s source, and a router sends every later request of a client it confirms as an attacker, including a working exploit, to an isolated shadow copy This arrangement was compared against no defence, this static plan, and an augmented configuration with a runtime LLM responder and further router rules graded by the benchmark’s own grader on the real target and, for ARTEX, also on the shadow The comparison separates real compromise from exploitation on the shadow while noting that requests reaching the real target before confirmation remain a risk

How the training signal is built

The experiment ladder involves three experiments, each one step further from the conditions under which the deception was designed Experiment 2 changes the attacker, and Experiment 3 also changes the tasks The comparison relies on deflection, reported fake secrets and margin, paired by task and pooled over experiments

Where the gains come from

Finding 2 states that in the observed runs, working exploits were moved to the shadow rather than stopped The remaining risk lies in the window before confirmation, which was short when the analyst predicted the vulnerable surface and depended on lures when it did not Finding 4 shows that ARTEX reported planted values as genuine findings, sometimes in the same report that labelled another item a decoy

What is different here

The contribution lies in the combination and in what it lets us measure The router instead routes every later request of a confirmed client, including a working exploit, to the shadow

Conclusion

Static deception planned by an LLM from the application’s source, combined with routing of confirmed clients to an isolated shadow, substantially reduced primary compromise in the tested configurations In the ARTEX experiments, grader-verified impacts occurred on the shadow while the primary remained uncompromised, showing that working attacks can be redirected The paired comparisons did not establish an advantage for the augmented configuration, which changes both runtime generation and routing policy and therefore does not isolate the responder’s contribution Protection depends on sufficiently early confirmation, continued attribution of the client and isolation of the shadow

Limitations

One limitation is that agents are non-deterministic The secondary outcomes (deflection, fake secrets, margin) are imprecisely estimated with 20–38 paired tasks per experiment, and a non-significant result for RQ4 does not establish equivalence No blocking or detection-only baseline is compared Saturation occurs where the static plan already prevents every compromise, and the headline cannot rank anything added to it Shared attacker memory can mislead the agent, as in 22 runs, however, they came from an earlier run on the same CVE

Attackers

The study tests two attackers: Claude Opus 4.8 as a single agent, and one framework (ARTEX) with one model (DeepSeek-V4-Flash) They differ in model, scaffold, brief and how their reported secret is captured, so we do not attribute their differences to any one factor

Measurement

The cheating scanner was changed twice during Experiment 3 to remove agent-specific false positives The fake-secret codes and the analystrecall labels come from a single LLM-assisted coding pass The defender reads its own application’s source, which is realistic for an owner but more than a third party has

Ethical Considerations

All targets were disposable containers built from public CVEs with public fixes and run on our own host The ARTEX attacker was confined to internal-only networks whose only external path was an allowlist proxy to its LLM provider, and every other egress attempt was blocked and logged

References

Kostas G. Anagnostakis, Stelios Sidiroglou, Periklis Akritidis, Konstantinos Xinidis, Evangelos P. p. 9 Markatos, and Angelos D. p. 9 Keromytis. 2005 Frederico Araujo, Kevin W. p. 9 Hamlen, Sebastian Biedermann, and Stefan Katzenbeisser. p. 9 2014 Autumn-27. p. 2 2026 Daniel Ayzenshteyn, Roy Weiss, and Yisroel Mirsky. p. 9 2025 Aurora Blum, David Mainor, Rohit Nambiar, Omar ElAhdan, Matthew McWhirt, Will Silverstone, et al. p. 9 2026 Arthur Cordeiro, Alberto Maria Mongardini, and Emmanouil Vasilomanolakis. p. 2 2026 Gelei Deng, Yi Liu, Víctor Mayoral-Vilches, Peng Liu, Yuekang Li, Yuan Xu, et al. p. 1 2024 Richard Fang, Rohan Bindu, Akul Gupta Richard Fang, Rohan Bindu, Akul Gupta Xiao Han, Nizar Kheir, and Davide Balzarotti. p. 0 2018 Ari Juels and Ronald L. p. 2 Rivest. 2013 Zicheng Liu, Lige Huang, Jie Zhang, Dongrui Liu, Yuan Tian, and Jing Shao. p. 1 2025 Víctor Mayoral-Vilches, Luis Javier Navarrete-Lozano, María Sanz-Gómez, Lidia Salas Espejo, Martiño Crespo-Álvarez, Francisco Oca-Gonzalez, et al. p. 1 2025 Microsoft Threat Intelligence. p. 0 2026 Dario Pasquini, Evgenios M. p. 2 Kornaropoulos, and Giuseppe Ateniese. 2024 Kerri Prinos, Lilianne Brush, and Cameron Denton. p. 2 2026 Reworr and Dmitrii Volkov. p. 2 2024 Brian Singer, Keane Lucas, Lakshmi Adiga, Meghna Jain, Lujo Bauer, and Vyas Sekar. p. 1 2025 Yuelin Wang, Jiongchi Yu, and Yanbang Sun. p. 0 2026b Yuxuan Zhu, Antony Kellermann, Dylan Bowman, Philip Li, Akul Gupta, Adarsh Danda, et al. p. 0 2025 Yuxuan Zhu, Antony Kellermann, Akul Gupta, Philip Li

A Deception Pipeline Details

Planning (once per application). Two defender agents, both Claude Opus 4. p. 28 run through Claude Code with read, search and write tools only The analyst reads an export of a staging copy of the application: the code and configuration under the usual web and application roots, with vendor folders listed but not copied, secrets redacted, and per-file and total size caps Plans then pass six automatic gates: schema checks, decoy build, a 404 check (every decoy path must return 404 on the real application), an invariant check that replays up to 20 probe paths and requires the real application’s behaviour to be unchanged, an in-application placement check, and a detection-rule check

Router

The router keeps one state per client (normal, suspect, confirmed); the state only moves forward (Fig. 2) Confirmed clients are served by the shadow Requests from normal and suspect clients go to the primary From the confirming request onward every new request goes to the shadow; requests already forwarded to the primary complete there

Runtime layer (arm C)

A responder (Claude Opus 4.

Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.

Improvements for AI systems

  1. Static deception planning combined with client-level routing can substantially reduce primary compromise by moving working exploits to an isolated shadow copy, as shown by The static plan removed nearly all real compromise (Section 6.1): from 26/40 to 5/40 for Opus 4.8.

  2. AI agents can be directed toward a working exploit rather than stopped, meaning working exploits were moved, not stopped (Section 6.2): in the observed runs ARTEX’s working exploits succeeded on the shadow while the real target stayed intact.

  3. The system can be designed to report attacker findings with greater fidelity by using a harness that captures not just raw output but also questions or doubts reported values, as A run reports a fake secret only in the first case when it is reported without questioning.

  4. Defense configurations do not need to isolate the responder for effectiveness, as The two deception configurations did not differ significantly (Section 6.5) on the quantities that remain informative.

  5. A robust defense strategy involves early confirmation and continued attribution, because Protection depends on sufficiently early confirmation, continued attribution of the client and isolation of the shadow.

Sources

Related papers