Evaluator Ensembles Under Reward Hacking: Covariance Geometry and Finite-Search Guarantees
Fariya Afrin, Ibne Farabi Shihab
cs.LG
Submitted: 2026-08-08
Updated: 2026-08-11
License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
Terminology
Sources
- Concrete Problems in AI Safety
- When Does Combining Language Models Help? A Co-Failure Ceiling on Routing, Voting, and Mixture-of-Agents Across 67 Frontier Models
- Helping or Herding? Reward Model Ensembles Mitigate but do not Eliminate Reward Hacking
- Uncertainty Estimation for Language Reward Models
- Great Models Think Alike and this Undermines AI Oversight
- Bayesian Active Learning for Classification and Preference Learning
- When LLM Judge Scores Look Good but Best-of-N Decisions Fail
- WARM: On the Benefits of Weight Averaged Reward Models
- Is Best-of-N the Best of Them? Coverage, Scaling, and Optimality in Inference-Time Alignment
- Correlated Errors in Large Language Models
- Nine Judges, Two Effective Votes: Correlated Errors Undermine LLM Evaluation Panels
- Replacing Judges with Juries: Evaluating LLM Generations with a Panel of Diverse Models
- CARE: Confounder-Aware Aggregation for Reliable LLM Evaluation
Related papers
- Polynomial-Augmented Neural Networks (PANNs) with Weak Orthogonality Constraints for Enhanced Function and PDE Approximation
- AIRL-S: Unifying Reinforcement Learning and Search-Based Test-Time Scaling via Adversarial Inverse Reinforcement Learning
- Transformers as Bayesian In-Context Experimenters: Smoothness-Adaptive Efficient ATE Estimation
- Convergence issues in Relational Concept Analysis based on AOC-posets
- Beliefs Beyond Posteriors: The Predictive Cost of ELBO Binding
- Understanding Diffusion Models via Ratio-Based Function Approximation with SignReLU Networks