CalibratedRubric: Task-Adaptive Rubric Banks for Open-Ended LLM Evaluation

arXiv:2607.29252 · cs.CL, cs.AI, cs.LG · Submitted 2026-07-31 · Read on arXiv

Mengting Chen, Yanshu Sun, Wanting Liang, Beidi Luan, Rui Sun, Dezhi Chen, Jing Li, Zuo Bai

cs.CL, cs.AI, cs.LG

Submitted: 2026-07-31

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Terminology

Calibration — 17× in this paper · explained in 7 episodes
Benchmark — 10× in this paper · explained in 2 episodes
Fidelity — 6× in this paper · explained in 2 episodes
Papers — 6× in this paper · explained in 3 episodes
AUC — 4× in this paper · explained in 2 episodes
Ablation — 4× in this paper · explained in 1 episode
LLM-as-a-Judge — 4× in this paper · explained in 3 episodes
Redundancy — 3× in this paper · explained in 2 episodes
Actionability — 2× in this paper · explained in 3 episodes
Conditional Independence — 2× in this paper · explained in 2 episodes
Consistency — 2× in this paper · explained in 3 episodes
Generator — 2× in this paper · explained in 2 episodes
LLM judge — 2× in this paper · explained in 2 episodes
Ablation Study — 1× in this paper · explained in 3 episodes
Accuracy — 1× in this paper · explained in 1 episode
Core — 1× in this paper · explained in 2 episodes
Multimodal Reasoning — 1× in this paper · explained in 5 episodes
Parsimony — 1× in this paper · explained in 2 episodes
Perplexity — 1× in this paper · explained in 3 episodes
Pruning — 1× in this paper · explained in 4 episodes
RAG — 1× in this paper · explained in 3 episodes
Retrieval-Augmented Generation (RAG) — 1× in this paper · explained in 18 episodes
Robustness — 1× in this paper · explained in 5 episodes
Selection Bias — 1× in this paper · explained in 4 episodes