HLE-Verified: A Systematic Verification and Structured Revision of Humanity's Last Exam
The episode analyzes the paper 'HLE-Verified,' which systematically audited Humanity's Last Exam (HLE), a benchmark for AI models. The authors found significant errors in the original dataset, leading to a revised version where model accuracy increased dramatically. The hosts conclude that benchmarks require continuous auditing and maintenance.