Estimating problem difficulty without ground truth using Large Language Model comparisons
LLM Compare is a new method for estimating problem difficulty without needing human labels or ground truth data. It uses an LLM to perform pairwise comparisons between problems and then calculates Bradley-Terry scores based on these outcomes. This approach creates a dynamic, continuous, and model-agnostic measure that aligns well with human judgments.