Plant, Persist, Trigger: Sleeper Attack on Large Language Model Agents
cs.AI
Submitted: 2026-05-27
Updated: 2026-10-05
Code: https://github.com/openai/openai-agents-python
License: http://creativecommons.org/licenses/by/4.0/
The gist: Large Language Model (LLM) agents remain vulnerable to safety threats from the external environment, where attackers inject adversarial content into external observations such as tool-returned data,
Terminology
Abstract
Large Language Model (LLM) agents remain vulnerable to safety threats from the external environment, where attackers inject adversarial content into external observations such as tool-returned data, webpages, or MCP context, causing harmful agentic behaviors such as unsafe actions or incorrect outputs. Existing studies typically focus on single-interaction attacks, where the agent observes adversarial content and immediately exhibits harmful behavior within one user request. However, we show that adversarial content can also persist across interactions served by the same agent, making such threats harder to detect and mitigate. Specifically, adversarial content may persist in the agent state, remain dormant across interactions, and later be activated by a benign user query. We formalize this type of safety threat as Sleeper Attack. To evaluate it, we construct a benchmark with 1,896 instances covering six real-world harmful outcomes, three attack strategies, and three agent state targets: session context, memory, and reusable skills. Experiments on seven strong open-source and closed-source LLMs show that state-of-the-art LLM agents remain vulnerable to Sleeper Attack, even when they achieve low attack success rates under a single-interaction baseline. Our code and data are available at https://anonymous.4open.science/r/skdvnfu23ihr9wdscnksf1asdffsaef.
Sources
- GPT-4 Technical Report
- Indirect Prompt Injections: Are Firewalls All You Need, or Stronger Benchmarks?
- AgentPoison: Red-teaming LLM Agents via Poisoning Memory or Knowledge Bases
- From Untrusted Input to Trusted Memory: A Systematic Study of Memory Poisoning Attacks in LLM Agents
- Memory Injection Attacks on LLM Agents via Query-Only Interaction
- AgentLeak: A Benchmark for Internal-Channel Privacy Leakage in Multi-Agent LLM Systems
- SkillTrojan: Backdoor Attacks on Skill-Based Agent Systems
- DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning
- SkillJect: Effectively Automating Skill-Based Prompt Injection for Skill-Enabled Agents
- DeepSeek-V3.2: Pushing the Frontier of Open Large Language Models
- AgentDoG: A Diagnostic Guardrail Framework for AI Agent Safety and Security
- ToolSafe: Enhancing Tool Invocation Safety of LLM-based agents via Proactive Step-level Guardrail and Feedback
- AgentSCOPE: Evaluating Contextual Privacy Across Agentic Workflows
- Hidden in Memory: Sleeper Memory Poisoning in LLM Agents
- Supply-Chain Poisoning Attacks Against LLM Coding Agent Skill Ecosystems
- Skill-Inject: Measuring Agent Vulnerability to Skill File Attacks
- MemoryGraft: Persistent Compromise of LLM Agents via Poisoned Experience Retrieval
- A Comprehensive Survey in LLM(-Agent) Full Stack Safety: Data, Training and Deployment
- From Storage to Steering: Memory Control Flow Attacks on LLM Agents
- Qwen3 Technical Report
Related papers
- MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
- Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
- Agents' Last Exam
- The Clinician's Veto: Navigating Trust, Liability, and Uncertainty in Autonomous AI Prescribing
- MindHelper: Closed-Loop Embodied Mental-State Reasoning for Precision Intervention
- Incumbent Advantage: Brand Bias and Cognitive Manipulation Dynamics in LLM Recommendation Systems