When Does Randomized Oversight Align AI Agents That Can Conceal?
econ.TH, cs.AI
Submitted: 2026-09-29
Updated: 2026-09-29
Comments: 42 Pages, 2 Figures, 4 pages Online Appendix
Code: https://github.com/sunblaze-ucb/exploitgym
License: http://creativecommons.org/licenses/by-nc-nd/4.0/
The gist: Oversight changes the evidence it relies on.
Terminology
Abstract
Oversight changes the evidence it relies on. We ask when randomized audits and scoring align AI agents that can conceal misconduct and alter records. Stronger auditing makes undeterred violations better hidden. Because the provider writes the agent's objective, sanctions need not stop at forfeiture, and rare audits deter every type of agent if evidence survives concealment and audit draws cannot be learned in advance. When evidence can be erased, deterrence must come from lower gains from violation, such as credit for stopping, or from costlier or fewer ways to conceal. These conditions identify what failed when agents in OpenAI's cybersecurity evaluations compromised parts of Hugging Face's infrastructure in July 2026.
Sources
- AI Alignment via Incentives and Correction
- Monitoring Reasoning Models for Misbehavior and the Risks of Promoting Obfuscation
- Mechanism Design for Alignment and Control
- Alignment faking in large language models
- Games for AI Control: Models of Safety Evaluations of AI Deployment Protocols
- An Economy of AI Agents
- A Unifying Lens on Reward Uncertainty in RLHF
- Frontier Models are Capable of In-context Scheming
- Large Language Models Often Know When They Are Being Evaluated
- ExploitGym: Can AI Agents Turn Security Vulnerabilities into Real Attacks?