When Safety Becomes an Attack Surface: Manipulating Abstention in Tool-Using AI Agents
cs.CR, cs.AI
Submitted: 2026-11-11
Updated: 2026-11-11
Terminology
Sources
- Safety-Tuned LLaMAs: Lessons From Improving the Safety of Large Language Models that Follow Instructions
- StruQ: Defending Against Prompt Injection with Structured Queries
- OR-Bench: An Over-Refusal Benchmark for Large Language Models
- AgentDojo: A Dynamic Environment to Evaluate Prompt Injection Attacks and Defenses for LLM Agents
- Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection
- Defending Against Indirect Prompt Injection Attacks With Spotlighting
- AgentAbstain: Do LLM Agents Know When Not to Act?
- Formalizing and Benchmarking Prompt Injection Attacks and Defenses
- The Data Provenance Initiative: A Large Scale Audit of Dataset Licensing & Attribution in AI
- Discovering Language Model Behaviors with Model-Written Evaluations
- Ignore Previous Prompt: Attack Techniques For Language Models
- XSTest: A Test Suite for Identifying Exaggerated Safety Behaviours in Large Language Models
- Identifying the Risks of LM Agents with an LM-Emulated Sandbox
- Toolformer: Language Models Can Teach Themselves to Use Tools
- Towards Understanding Sycophancy in Language Models
- The Instruction Hierarchy: Training LLMs to Prioritize Privileged Instructions
- Large Language Models are not Fair Evaluators
- ReAct: Synergizing Reasoning and Acting in Language Models
- $\tau$-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domains
- Benchmarking and Defending Against Indirect Prompt Injection Attacks on Large Language Models
Related papers
- From Strategy Search to Policy Learning: MCTS-Guided Adversarial Co-Evolution for LLM Safety Alignment
- Privacy Leakage from Recurrent State in LLM Inference
- Incrementally Verifiable Oracle Aided Computation
- Privilege and confidentiality in generative AI workflows
- BadMoE: Backdooring Mixture-of-Experts LLMs via Routing-Aware Triggers and Task-Underutilized Experts
- CoVault: Secure, Scalable Analytics of Personal Data