CoER: Defending against Adaptive Indirect Prompt Injection via Adversarial Co-Evolution and Refinement

arXiv:2609.07529 · cs.LG · Submitted 2026-09-07 · Read on arXiv

cs.LG

Submitted: 2026-09-07

Updated: 2026-09-30

Comments: 31 pages, 7 figures

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Terminology

• Verifier — 2× in this paper · explained in 2 episodes
• Ablation — 1× in this paper · explained in 1 episode
• Benchmark — 1× in this paper · explained in 2 episodes

Sources

• ChatInject: Abusing Chat Templates for Prompt Injection in LLM Agents
• Meta-SecAlign: Training LLMs against Prompt Injection for Robust Agents
• Learning to Inject: Automated Prompt Injection via Reinforcement Learning
• Defeating Prompt Injections by Design
• Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection
• Defending against Adaptive Prompt Injection Attacks via Reasoning-enabled Task Alignment
• Defending Against Indirect Prompt Injection Attacks With Spotlighting
• AgentLAB: Benchmarking LLM Agents against Long-Horizon Attacks
• AgentDyn: Are Your Agent Security Defenses Deployable in Real-World Dynamic Environments?
• SafeEvolve: Harness-Policy Co-Evolution from Agent Experience for Safety Alignment
• Proximal Policy Optimization Algorithms
• Progent: Securing AI Agents with Privilege Control
• The Instruction Hierarchy: Training LLMs to Prioritize Privileged Instructions
• GPT-Red: Automated Red Teaming via Self-Play at Scale
• Adversarial Reinforcement Learning for Large Language Model Agent Safety
• MAGIC: A Co-Evolving Attacker-Defender Adversarial Game for Robust LLM Safety
• PISmith: Reinforcement Learning-based Red Teaming for Prompt Injection Defenses
• AgentAlign: Navigating Safety Alignment in the Shift from Informative to Agentic Large Language Models

Related papers

• Polynomial-Augmented Neural Networks (PANNs) with Weak Orthogonality Constraints for Enhanced Function and PDE Approximation
• AIRL-S: Unifying Reinforcement Learning and Search-Based Test-Time Scaling via Adversarial Inverse Reinforcement Learning
• Transformers as Bayesian In-Context Experimenters: Smoothness-Adaptive Efficient ATE Estimation
• Convergence issues in Relational Concept Analysis based on AOC-posets
• Beliefs Beyond Posteriors: The Predictive Cost of ELBO Binding
• Understanding Diffusion Models via Ratio-Based Function Approximation with SignReLU Networks