Latency-Response Theory Model: Evaluating Large Language Models via Response Accuracy and Chain-of-Thought Length

arXiv:2512.07019 · stat.ME, cs.AI, stat.AP, stat.ML · Submitted 2025-12-07 · Read on arXiv

stat.ME, cs.AI, stat.AP, stat.ML

Submitted: 2025-12-07

Updated: 2026-08-31

Code: https://github.com/Toby-X/Latency-Response-Theory-Model

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Terminology

• Accuracy — 58× in this paper · explained in 1 episode
• Benchmark — 29× in this paper · explained in 2 episodes
• Fisher Information — 11× in this paper · explained in 3 episodes
• Item Response Theory — 8× in this paper
• Computerized Adaptive Testing — 7× in this paper
• Consistency — 7× in this paper · explained in 3 episodes
• Benchmarking — 4× in this paper · explained in 3 episodes
• Large Language Models (LLMs) — 3× in this paper · explained in 6 episodes
• Robustness — 3× in this paper · explained in 5 episodes
• Chain-of-Thought (CoT) — 2× in this paper · explained in 4 episodes
• Hallucination — 2× in this paper · explained in 3 episodes
• Hierarchical Framework — 2× in this paper · explained in 2 episodes
• Interpretability — 2× in this paper · explained in 15 episodes
• Active Learning — 1× in this paper · explained in 6 episodes
• Algorithmic Stability — 1× in this paper · explained in 1 episode
• Chain-of-Thought (CoT) Prompting — 1× in this paper · explained in 3 episodes
• Chain-of-Thought Prompting — 1× in this paper · explained in 2 episodes
• Generative Model — 1× in this paper · explained in 2 episodes
• Hallucination Detection — 1× in this paper · explained in 2 episodes
• Latent Representation — 1× in this paper · explained in 2 episodes
• Marginal Maximum Likelihood — 1× in this paper
• Reinforcement Learning — 1× in this paper · explained in 4 episodes
• Verifier — 1× in this paper · explained in 2 episodes

Sources

• gpt-oss-120b & gpt-oss-20b Model Card
• Evaluating Large Language Models Trained on Code
• Training Verifiers to Solve Math Word Problems
• DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning
• OpenAI o1 System Card

Related papers

• Doubly robust inference via calibration
• Bayesian Empirical Bayes: Simultaneous Inference from Probabilistic Symmetries
• Weak-Form Recovery of Stochastic Generators
• Optimal Watermark Localization in Mixed-Source Large Language Model Texts
• Optimized Conformal Selection: Powerful Selective Inference After Data-Dependent Score Construction
• Small area learning