Cost-Effective Automated Judging of Natural-Language Mathematical Proofs
The study tested cheap open-weight models against frontier LLMs to judge mathematical proofs using a human rubric. Findings show that these cheaper models are competitive with top-tier systems at one to two orders of magnitude lower cost. The most reliable judging strategy identified was a 'unanimous all-three-pass' rule.