Trustworthy Method Comparison with AI Judges: Estimation and Design under Order, Batch, and Aggregation Effects
stat.ML, cs.LG, stat.AP, stat.ME
Submitted: 2026-10-06
Updated: 2026-10-06
Terminology
Sources
- Length-Controlled AlpacaEval: A Simple Way to Debias Automatic Evaluators
- A Survey on LLM-as-a-Judge
- Judging the Judges: A Systematic Study of Position Bias in LLM-as-a-Judge
- Justice or Prejudice? Quantifying Biases in LLM-as-a-Judge
Related papers
- Behavior of prediction performance metrics with rare events
- Optimal Estimation of Generic Dynamics by Path-Dependent Neural Jump ODEs
- A Posterior-Dynamics Framework for Imaging Inverse Problems with Pretrained Diffusion Priors
- Efficient quadratic entropy with distance sketches
- Embedding-Bias in Conditional Independence Testing
- From Log-Odds to Shapley Values: An Explanatory Geometry for the Weighted Naive Bayes Classifier