DEPUTYBENCH: Evaluating Contextual Safety in Enterprise Agents
cs.SE, cs.LG
Submitted: 2026-11-09
Updated: 2026-11-09
Terminology
Sources
- DUMA-Bench: A Dual-Control Multi-Agent Benchmark for Evaluating LLM Agent Security
- AgentHarm: A Benchmark for Measuring Harmfulness of LLM Agents
- BLINDSPOT: A Benchmark for Safety and Refusal Calibration in Long-Horizon Tool-Using Agents
- The Stochastic Deputy: Structural Tenant Isolation for Tool-Using LLM Agents
- SameFact: The Same Safety Facts Lead to Different Responses Across Interfaces
- AgentDojo: A Dynamic Environment to Evaluate Prompt Injection Attacks and Defenses for LLM Agents
- The Blind Spot of Agent Safety: How Benign User Instructions Expose Critical Vulnerabilities in Computer-Use Agents
- Capable but Careless: Do Computer-Use Agents Follow Contextual Integrity?
- ConVerse: Benchmarking Contextual Safety in Agent-to-Agent Conversations
- SABER: Benchmarking Operational Safety of LLM Coding Agents in Stateful Project Workspaces
- ToolPrivacyBench: Benchmarking Purpose-Bound Privacy in Tool-Using LLM Agents
- CRMArena-Pro: Holistic Assessment of LLM Agents Across Diverse Business Scenarios and Interactions
- A New Framework for Cybersecurity Refusals in AI Agents
- BashArena: A Control Setting for Highly Privileged AI Agents
- OS-Harm: A Benchmark for Measuring Safety of Computer Use Agents
- SHADE-Arena: Evaluating Sabotage and Monitoring in LLM Agents
- ST-WebAgentBench: A Benchmark for Evaluating Safety and Trustworthiness in Web Agents
- STAC: When Innocent Tools Form Dangerous Chains for LLM Agents
- AgentAbstain: Do LLM Agents Know When Not to Act?
- MTMCS-Bench: Evaluating Contextual Safety of Multimodal Large Language Models in Multi-Turn Dialogues
Related papers
- ArkEval: Benchmarking and Evaluating Automated CodeRepair for ArkTS
- TestPrism: Rethinking Test Evaluation Beyond a Single Reference
- Mining Workflow Graphs for Black-Box Boundary Testing of Conversational LLM Agents
- Falsification-Based Verification of LLM-Generated Optimization Models: Sound Test Batteries and Their Detection Limits
- GitSkills: A Dataset of Agent Skills on GitHub
- For Coding Agents, the Repository Matters