CLM-as-a-Judge: Evaluating an Open Contrastive Decision Model on Public Judge Benchmarks
cs.LG, cs.AI, cs.CL
Submitted: 2026-10-05
Updated: 2026-10-05
Code: https://github.com/ml-explore/mlx
Terminology
Sources
- FrugalGPT: How to Use Large Language Models While Reducing Cost and Improving Performance
- BERT-as-a-Judge: A Robust Alternative to Lexical Methods for Efficient Reference-Based LLM Evaluation
- On Calibration of Modern Neural Networks
- Who's Your Judge? On the Detectability of LLM-Generated Judgments
- HaluEval: A Large-Scale Hallucination Evaluation Benchmark for Large Language Models
- JEV-as-a-Judge: Accept When Confident, Escalate When Unsure
- Skywork-Reward: Bag of Tricks for Reward Modeling in LLMs
- Skywork-Reward-V2: Scaling Preference Data Curation via Human-AI Synergy
- RM-Bench: Benchmarking Reward Models of Language Models with Subtlety and Style
- RewardBench 2: Advancing Reward Model Evaluation
- Laya as a Typed Probabilistic Assessor: An Independent Reproduction and a Preregistered Study of Calibration and Selective Escalation
- LLM Evaluators Recognize and Favor Their Own Generations
- JudgeBench: A Benchmark for Evaluating LLM-based Judges
- Large Language Models are not Fair Evaluators
- Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena
Related papers
- Polynomial-Augmented Neural Networks (PANNs) with Weak Orthogonality Constraints for Enhanced Function and PDE Approximation
- AIRL-S: Unifying Reinforcement Learning and Search-Based Test-Time Scaling via Adversarial Inverse Reinforcement Learning
- Transformers as Bayesian In-Context Experimenters: Smoothness-Adaptive Efficient ATE Estimation
- Convergence issues in Relational Concept Analysis based on AOC-posets
- Beliefs Beyond Posteriors: The Predictive Cost of ELBO Binding
- Understanding Diffusion Models via Ratio-Based Function Approximation with SignReLU Networks