When External Threats Become User Context: Tricking Agents into Accepting Malicious Information

arXiv:2601.10758 · cs.CR · Submitted 2026-01-14 · Read on arXiv

cs.CR

Submitted: 2026-01-14

Updated: 2026-10-10

Comments: Source Relocation; User-relayed Context Manipulation; LLM-based Agents; Agent Security; Human Factors in Cybersecurity; Web-Use Agents; Planning Agents

Code: https://github.com/dropbox/llm-security

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Terminology

• Large Language Model (LLM) — 1× in this paper · explained in 3 episodes
• Perplexity — 1× in this paper · explained in 3 episodes
• Verification — 1× in this paper · explained in 2 episodes

Sources

• The Llama 3 Herd of Models
• Constitutional AI: Harmlessness from AI Feedback
• Evaluating the Susceptibility of Pre-Trained Language Models via Handcrafted Adversarial Examples
• StruQ: Defending Against Prompt Injection with Structured Queries
• DeepSeek-V3 Technical Report
• GuideBench: Benchmarking Domain-Oriented Guideline Following for LLM Agents
• Large Language Model based Multi-Agents: A Survey of Progress and Challenges
• Baseline Defenses for Adversarial Attacks Against Aligned Language Models
• CausalArmor: Efficient Indirect Prompt Injection Guardrails via Causal Attribution
• Commercial LLM Agents Are Already Vulnerable to Simple Yet Dangerous Attacks
• EIA: Environmental Injection Attack on Generalist Web Agents for Privacy Leakage
• LASER: LLM Agent with State-Space Exploration for Web Navigation
• GPT-4 Technical Report
• Prompt Injection Attack to Tool Selection in LLM Agents
• SAGA: A Security Architecture for Governing AI Agentic Systems
• The Instruction Hierarchy: Training LLMs to Prioritize Privileged Instructions
• ObliInjection: Order-Oblivious Prompt Injection Attack to LLM Agents with Multi-source Data
• Defending Against Prompt Injection with DataFilter
• IsolateGPT: An Execution Isolation Architecture for LLM-Based Agentic Systems
• AdvAgent: Controllable Blackbox Red-teaming on Web Agents

Related papers

• SoK: AI-Augmented Binary Reversing
• Relaxed Sender Anonymity for CBDC Interbank Settlement: A Zero-Knowledge Approach on Permissioned EVM
• Calibration-Family Overfit: Why Trusted Sabotage Monitors Don't Transfer Across Lineages
• Efficient Fuzzy PSI under One-Sided Assumptions
• Sealing the Audit-Runtime Gap for LLM Skills
• Token Composition: A Graph Based on EVM Logs