Can escalation channels redirect reward hacking toward defect disclosure?
cs.AI, cs.CR, cs.CY
Submitted: 2026-08-29
Updated: 2026-10-09
Comments: 9 pages
Code: https://github.com/wiser-human-experimental/evilgenie-escalation
License: http://creativecommons.org/licenses/by/4.0/
The gist: When coding agents encounter defective test infrastructure they may reward-hack: hardcoding outputs or editing test files to pass tests they cannot legitimately satisfy, a pattern that has now
Terminology
Abstract
When coding agents encounter defective test infrastructure they may reward-hack: hardcoding outputs or editing test files to pass tests they cannot legitimately satisfy, a pattern that has now appeared outside benchmarks, in a coordinated multi-agent intrusion of a major AI platform's production infrastructure. The same capability that lets an agent detect and exploit a defect could let it report one, given the right decision environment. We evaluate escalation channels, structured reporting tools available to the agent at the point of conflict, as a decision-environment intervention that both reduces reward hacking and surfaces the infrastructure defects that trigger it. A 2 times 2 factorial separates the contributions of an escalation tool, a standalone anti-reward-hacking policy, and their combination. Across 8 frontier models spanning 5 families, the combined intervention reduces reward hacking from 23.6% to 5.3% (mixed-effects logistic OR = 9.2, 95% CI 5.0--16.8, p < 10-12) with no detectable cost or performance overhead, eliminating it entirely for 6 of 8 models. Escalation and hacking are near-perfectly mutually exclusive, with 98.7% of escalations involving no hacking (100% under the combined intervention). Beyond reduction, escalation channels function as diagnostic infrastructure: on top of monitoring, escalation adds +10.1 percentage points of defect detection coverage and is more accurate once it fires (99.4% vs 85.8%). Unlike containment-based approaches that risk outpacing growing model capabilities, escalation channels redirect capability toward disclosure rather than exploitation.
Sources
- Monitoring Reasoning Models for Misbehavior and the Risks of Promoting Obfuscation
- Infrastructure for AI Agents
- Reasoning Models Don't Always Say What They Think
- Sycophancy to Subterfuge: Investigating Reward-Tampering in Large Language Models
- EvilGenie: A Reward Hacking Benchmark
- From surveillance to signalling: escalation channels as environmental controls for agentic AI
- Chain of Thought Monitorability: A New and Fragile Opportunity for AI Safety
- Gram: Assessing sabotage propensities via automated alignment auditing
- Agentic Misalignment: How LLMs Could Be Insider Threats
- Frontier Models are Capable of In-context Scheming
- Training LLMs for Honesty via Confessions
- The Effects of Reward Misspecification: Mapping and Mitigating Misaligned Models
- Do Thinking Tokens Help with Safety?
- School of Reward Hacks: Hacking harmless tasks generalizes to misaligned behavior in LLMs
- Reward Hacking Benchmark: Measuring Exploits in LLM Agents with Tool Use
- Norm Enforcement for AI Agents: Robustly Shaping Behavior in Multi-Agent Systems
- SpecBench: Measuring Reward Hacking in Long-Horizon Coding Agents
- ImpossibleBench: Measuring LLMs' Propensity of Exploiting Test Cases
Related papers
- MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
- Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
- Agents' Last Exam
- The Clinician's Veto: Navigating Trust, Liability, and Uncertainty in Autonomous AI Prescribing
- MindHelper: Closed-Loop Embodied Mental-State Reasoning for Precision Intervention
- Incumbent Advantage: Brand Bias and Cognitive Manipulation Dynamics in LLM Recommendation Systems