Breaking Bad: "Going Badly" Activations Drive Reward Hacking
cs.CL, cs.AI
Submitted: 2026-11-11
Updated: 2026-11-11
Terminology
Sources
- Monitoring Reasoning Models for Misbehavior and the Risks of Promoting Obfuscation
- Monitoring and Discovering Reward Hacking with Internal Representations during LLM Evaluations
- Stories in Space: In-Context Learning Trajectories in Conceptual Belief Space
- Persona Vectors: Monitoring and Controlling Character Traits in Language Models
- Reasoning Models Don't Always Say What They Think
- "Did you lie?" Evaluating Lie Detectors across Model Scale and Belief-Verified Model Organisms
- Sycophancy to Subterfuge: Investigating Reward-Tampering in Large Language Models
- Scaling Laws for Reward Model Overoptimization
- Detecting Strategic Deception Using Linear Probes
- How's it going? Reinforcement learning in language models recruits a functional welfare axis
- Functional Welfare for AI Safety
- The Value Axis: Language Models Encode Whether They're on the Right Track
- Steering Language Model Goals with Value Transplant
- Measuring AI Ability to Complete Long Software Tasks
- TACO: Topics in Algorithmic COde generation dataset
- Natural Emergent Misalignment from Reward Hacking in Production RL
- CheatBench: Measuring Reward Gaming in AI Agents
- The Pain Axis: LLMs Represent Self-Directed Harm and Act on It
- Inoculation Prompting: Eliciting traits from LLMs during training can suppress them at test-time
- Steering Language Models With Activation Engineering
Related papers
- ALARM: Audio-Language Alignment for Reasoning Models
- ThinkSum: Probabilistic reasoning over sets using large language models
- Coherence boosting: When your pretrained language model is not paying enough attention
- From Facts to Insights: A Persona-Driven Dual Memory Framework and Dataset for Role-Playing Agents
- Improving General Role-Playing Agents via Psychology-Grounded Reasoning and Role-Aware Policy Optimization
- The Art of Asking: Multilingual Prompt Optimization for Synthetic Data