Correct Looks Better: Pairwise Comparisons Reveal Accuracy Rankings
The study tested if rankings derived from pairwise comparisons of generative model answers match accuracy rankings when ground truth is available. Findings show strong agreement, with Bradley-Terry rankings achieving high correlation and recovering accuracy order across five benchmarks. Style and judge biases have only minor effects on the final results.