AGORA: Reliability-Aware Aggregation of LLM Judges on Small Evaluation Sets
cs.AI, cs.CL
Submitted: 2026-11-03
Updated: 2026-11-11
Terminology
Sources
- Dependence-Aware Label Aggregation for LLM-as-a-Judge via Ising Models
- Diagnosing the Reliability of LLM-as-a-Judge via Item Response Theory
- Who can we trust? LLM-as-a-jury for Comparative Assessment
- Replacing Judges with Juries: Evaluating LLM Generations with a Panel of Diverse Models
- Mixture-of-Agents Enhances Large Language Model Capabilities
- CARE: Confounder-Aware Aggregation for Reliable LLM Evaluation
Related papers
- Multi-Objective Structured Pruning of LLMs for Latency and Model Size Optimization
- MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
- Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
- Embeddings for Preferences, Not Semantics
- Can We Trust Item Response Theory for AI Evaluation?
- Diagnosing Task Insensitivity in Language Agents