Goodhart in the Planner: Sampling-Based Control Exploits Rewards Learned from Human Preferences
cs.RO, cs.AI, cs.LG
Submitted: 2026-11-06
Updated: 2026-11-17
Terminology
Sources
- Revisiting scaling laws for reward optimization
- Theoretical guarantees on the best-of-n alignment policy
- Imperfect World Models are Exploitable
- Active Learning MPC Objective Functions from Preferences
- D4RL: Datasets for Deep Data-Driven Reinforcement Learning
- RoboMonkey: Scaling Test-Time Sampling and Verification for Vision-Language-Action Models
- TD-M(PC)$^2$: Improving Temporal Difference MPC Through Policy Constraint
- CriticHack: Evaluating Visual Rewards Under Robot Policy Optimization
- Categorizing Variants of Goodhart's Law
- UBP2: Uncertainty-Balanced Preference Planning for Efficient Preference-based Reinforcement Learning
- Residual-Conservative Model Predictive Path Integral Control
Related papers
- Rethinking Actions: Continuity-Aware Priors for World-Model Planning
- MOSAIC: Modular Obstacle and Spatial-Aware Framework for Manipulation In Cluttered Scenes
- SAGAS: Semantic-Aware Graph-Assisted Stitching for Offline Temporal Logic Planning
- DexSeed: Expanding Human Hand-Object Interactions into Robot Demonstrations for Dexterous Policy Learning
- LaST0.5: A Generalist World Action Model with Interaction-Centric Latent Reasoning
- CIALLO: Learning Shared Intent Representations for Cross-Embodiment Manipulation