Designing Effective Monitor-Based Interventions for Mitigating Reward Hacking During RL
cs.LG
Submitted: 2026-11-06
Updated: 2026-11-06
Code: https://github.com/ariahw/rl-rewardhacking-ext
Terminology
Sources
- Concrete Problems in AI Safety
- Recontextualization Mitigates Specification Gaming without Modifying the Specification
- Monitoring Reasoning Models for Misbehavior and the Risks of Promoting Obfuscation
- Persona Vectors: Monitoring and Controlling Character Traits in Language Models
- Sycophancy to Subterfuge: Investigating Reward-Tampering in Large Language Models
- RL-Obfuscation: Can Language Models Learn to Evade Latent-Space Monitors?
- Knowledge-Level Consistency Reinforcement Learning: Dual-Fact Alignment for Long-Form Factuality
- Natural Emergent Misalignment from Reward Hacking in Production RL
- The Effects of Reward Misspecification: Mapping and Mitigating Misaligned Models
- Features as Rewards: Scalable Supervision for Open-Ended Tasks via Interpretability
- DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models
- HybridFlow: A Flexible and Efficient RLHF Framework
- Emotion Concepts and their Function in a Large Language Model
- Inoculation Prompting: Eliciting traits from LLMs during training can suppress them at test-time
- Inoculation Prompting: Instructing LLMs to misbehave at train-time improves test-time alignment
- DAPO: An Open-Source LLM Reinforcement Learning System at Scale
- ImpossibleBench: Measuring LLMs' Propensity of Exploiting Test Cases
Related papers
- A Physics-Regulated Neural Framework for Learning 3D Grain Growth Dynamics
- FlexMoE: Learning One-for-All Multi-Budget Expert Pruning for Flexible MoE Deployment
- OptiMind: Improving Small Language Models for Optimization Modeling through Formulation Guidance and Multi-Turn Learning
- Staying in the Sweet Spot: Boost Reasoning Evolution via Responsive Rollout Difficulty Manipulation
- Examining the Impact of Data and Training Methods on Emergent and Subliminal Misalignment
- Rethinking Black-Box On-Policy Distillation: A Rubric-based Analysis