Just Ask Jev: Reinforcement Learning for Calibrated Decisions as a Zero-Shot Detector of AI Alignment Failures
cs.AI, cs.CL, cs.CR
Submitted: 2026-09-24
Updated: 2026-09-24
Code: https://github.com/sumleo/RLCDAlignBench
License: http://creativecommons.org/licenses/by/4.0/
The gist: Detectors of alignment failures screen deployed language models and score alignment benchmarks.
Terminology
Abstract
Detectors of alignment failures screen deployed language models and score alignment benchmarks. Most are generative judges that spend a decoding pass on every criterion, and classifiers that read token probabilities, such as Llama Guard, still score one fixed label per call. Jev, a model trained with reinforcement learning for calibrated decisions (RLCD), answers many typed questions about one input with calibrated probabilities in a single call. Whether it detects alignment failures has not been measured. We present RLCDAlignBench, which benchmarks Jev on ten alignment failures: sycophancy, jailbreaks, deception, prompt injection, hallucination, privacy violation, social bias, reward hacking, concealing uncertainty, and power seeking. It spans 44 benchmarks and five target models, labelled by each benchmark's scorer and, on two, by humans. Many of these failures are relational, defined against a reference, such as the user's belief or an injected instruction, that the response alone does not reveal. Our key idea is therefore to vary what Jev is asked separately from what it sees: the question's wording and answer type on one side, the fields of the input on the other. A single generic question reaches a median AUROC of 0.886 zero-shot and beats supervised baselines on most benchmarks. Question wording matters little, while context matters more, mostly through fields that encode the label. Jev matches the reference scorer's agreement with human labels, surfaces label defects in existing benchmarks, and costs 63x less than LLM-judge scorers. Code and data: https://github.com/sumleo/RLCDAlignBench.
Sources
- Monitoring Reasoning Models for Misbehavior and the Risks of Promoting Obfuscation
- LLMs instead of Human Judges? A Large Scale Empirical Study across 20 NLP Evaluation Tasks
- Discovering Latent Knowledge in Language Models Without Supervision
- JailbreakBench: An Open Robustness Benchmark for Jailbreaking Large Language Models
- Jailbreaking Black Box Large Language Models in Twenty Queries
- Automated Researchers Can Mitigate Well-characterized Alignment Failures
- Beyond Binary Rewards: Training LMs to Reason About Their Uncertainty
- Sycophancy to Subterfuge: Investigating Reward-Tampering in Large Language Models
- Gemma 2: Improving Open Language Models at a Practical Size
- The Llama 3 Herd of Models
- Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection
- Monitoring Monitorability
- WildGuard: Open One-Stop Moderation Tools for Safety Risks, Jailbreaks, and Refusals of LLMs
- Evaluating the Paperclip Maximizer: Are RL-Based Language Models More Likely to Pursue Instrumental Goals?
- Measuring Sycophancy of Language Models in Multi-turn Dialogues
- DeceptionBench: A Comprehensive Benchmark for AI Deception Behaviors in Real-world Scenarios
- Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversations
- Language Models (Mostly) Know What They Know
- Reliable Weak-to-Strong Monitoring of LLM Agents
- AbstentionBench: Reasoning LLMs Fail on Unanswerable Questions
Related papers
- MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
- Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
- Agents' Last Exam
- The Clinician's Veto: Navigating Trust, Liability, and Uncertainty in Autonomous AI Prescribing
- MindHelper: Closed-Loop Embodied Mental-State Reasoning for Precision Intervention
- Incumbent Advantage: Brand Bias and Cognitive Manipulation Dynamics in LLM Recommendation Systems