Widening Harm Evaluation Coverage with a Judge that Reads the World
cs.AI
Submitted: 2026-11-02
Updated: 2026-11-02
Code: https://github.com/UKGovernmentBEIS/control-arena
Terminology
Sources
- AgentHarm: A Benchmark for Measuring Harmfulness of LLM Agents
- How does information access affect LLM monitors' ability to detect sabotage?
- CoT Red-Handed: Stress Testing Chain-of-Thought Monitoring
- Monitoring Reasoning Models for Misbehavior and the Risks of Promoting Obfuscation
- SmartSnap: Proactive Evidence Seeking for Self-Verifying Agents
- DecodingTrust-Agent Platform (DTap): A Controllable and Interactive Red-Teaming Platform for AI Agents
- Agentic Reward Modeling: Verifying GUI Agent via Progressive Trajectory-Grounded Interaction
- ProRe: A Proactive Reward System for GUI Agents via Reasoner-Actor Collaboration
- AgentDojo: A Dynamic Environment to Evaluate Prompt Injection Attacks and Defenses for LLM Agents
- Safety Testing LLM Agents at Scale: From Risk Discovery to Evidence-Grounded Verification
- AI Control: Improving Safety Despite Intentional Subversion
- SABER: Benchmarking Operational Safety of LLM Coding Agents in Stateful Project Workspaces
- AI Agents That Matter
- Log analysis is necessary for credible evaluation of AI agents
- OS-Harm: A Benchmark for Measuring Safety of Computer Use Agents
- SHADE-Arena: Evaluating Sabotage and Monitoring in LLM Agents
- Taxonomy and Consistency Analysis of Safety Benchmarks for AI Agents
- SLEIGHT-Bench: A Benchmark of Evasion Attacks Against Agent Monitors
- Autonomous Evaluation and Refinement of Digital Agents
- The Art of Building Verifiers for Computer Use Agents
Related papers
- Multi-Objective Structured Pruning of LLMs for Latency and Model Size Optimization
- MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
- Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
- Embeddings for Preferences, Not Semantics
- Can We Trust Item Response Theory for AI Evaluation?
- Diagnosing Task Insensitivity in Language Agents