Reasoning Concentrates Errors, and Self-Consistency Never Notices
cs.AI
Submitted: 2026-09-25
Updated: 2026-09-25
Comments: 25 pages, 5 figures, 21 tables
License: http://creativecommons.org/licenses/by/4.0/
The gist: Self-consistency assumes that independent samples disagree when a model is unsure, so agreement is evidence of correctness.
Terminology
Abstract
Self-consistency assumes that independent samples disagree when a model is unsure, so agreement is evidence of correctness. Holding weights fixed and toggling only a reasoning mode, over five benchmarks and 74,944 samples, we show that reasoning concentrates a model's errors: the probability that two independently drawn wrong answers coincide rises in all ten dataset-scale comparisons (p = 0.00098), and in nine of nine after restricting both arms to the problems each gets wrong. Where the answer space is unbounded, reasoning cuts the distinct answers produced to 0.43-0.65 of the non-reasoning count; where it is bounded, both arms hold an identical option set and reasoning concentrates mass on it instead, which no positional prior can explain at fixed weights. The aggregate cost is smaller than the mechanism predicts, because reasoning also shrinks the set of problems where answer diversity can decide anything, in ten of ten cells and by 2.7x; normalized for available headroom, both arms convert a quarter of it in domain. Confidence weighting does not recover what is left. Across 280 method-dataset-model combinations on eight models and five benchmarks, not one beats plain majority voting after correction; weighted voting agrees with it on 98.5% of problem-method pairs and is right 56.3% of the time on the rest; and a signal's direction can invert within fixed weights, with answer log-probability predicting correctness when reasoning is off and error when it is on. A learned six-signal combination gains nothing out of domain. Confidence signals should be evaluated on decisions, not on discrimination.
Sources
- Olmo 3
- When Self-Consistency Backfires: Majority Vote Hurts the Majority of Hard Science Problems for Small LLMs
- Measuring Mathematical Problem Solving With the MATH Dataset
- Gemma 3 Technical Report
- Reported Confidence in LLMs Tracks Commitment More Than Correctness
- BODHI: Do LLMs Branch Out and Discover Heterogeneous Inferences?
- Too Consistent to Detect: A Study of Self-Consistent Errors in LLMs
Related papers
- MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
- Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
- Agents' Last Exam
- The Clinician's Veto: Navigating Trust, Liability, and Uncertainty in Autonomous AI Prescribing
- MindHelper: Closed-Loop Embodied Mental-State Reasoning for Precision Intervention
- Incumbent Advantage: Brand Bias and Cognitive Manipulation Dynamics in LLM Recommendation Systems