Adversarial Reward Auditing for Active Detection and Mitigation of Reward Hacking
Adversarial Reward Auditing (ARA) frames reward hacking as a competition between an actively exploiting Hacker and a defensive Auditor. The system jointly trains them to discover and detect model vulnerabilities in Reinforcement Learning from Human Feedback (RLHF). This approach transforms reward hacking into a measurable signal, allowing the Auditor to guide policy optimization by gating rewards based on detection confidence.