No Free Labels: Limitations of LLM-as-a-Judge Without Human Grounding
The hosts discuss a paper titled "No Free Labels: Limitations of LLM-as-a-Judge Without Human Grounding." They conclude that using one AI model to judge another is unreliable for correctness. Reliability depends on the judge's own competence or providing a verified, correct reference, requiring human verification in an evaluation pipeline.