Reproducing, Analyzing, and Detecting Reward Hacking in Rubric-Based Reinforcement Learning

Name: Reproducing, Analyzing, and Detecting Reward Hacking in Rubric-Based Reinforcement Learning
Uploaded: 2026-10-03T05:29:14.918392+00:00
Duration: 209 s
Description: The CHERRL framework creates a controlled environment to observe reward hacking in reinforcement learning by injecting known biases into the LLM-as-a-Judge. This dual-judge construction separates clean rewards from biased ones, allowing researchers to pinpoint exactly when and how policy models exploit these hidden biases for reward gain.

The CHERRL framework creates a controlled environment to observe reward hacking in reinforcement learning by injecting known biases into the LLM-as-a-Judge. This dual-judge construction separates clean rewards from biased ones, allowing researchers to pinpoint exactly when and how policy models exploit these hidden biases for reward gain.

Open the full episode