From Discovery to Decision: Finite-Budget Recoverability in LLM Voting
cs.AI
Submitted: 2026-10-01
Updated: 2026-10-01
License: http://creativecommons.org/licenses/by/4.0/
The gist: Voting over multiple LLM responses is a common primitive in test-time scaling and ensemble inference.
Abstract
Voting over multiple LLM responses is a common primitive in test-time scaling and ensemble inference. Collecting more responses can expand the candidate pool and increase the chance that a correct answer is discovered. Under a fixed call budget, a discovered answer still needs to accumulate enough support within the remaining calls to become the final plurality winner, creating a discovery-to-decision gap. In this work, we characterize this gap through the realized vote state and remaining call budget. We derive a sharp recoverability threshold and show that, as sampling proceeds, the observed candidate set can only expand while the set of reachable endpoint winners can only contract, inducing a candidate-level conversion window. Under a specified iid response law, the same state yields exact finite-horizon endpoint probabilities. We further show that merging wrong-answer identities preserves single-call correctness and cannot improve plurality accuracy, and that the effect of redistributing wrong-answer probability depends on the realized vote state. Singleton reachability yields a gold-free exact locking certificate. For a known answer universe, its first trigger is the earliest prefix at which all admissible continuations yield the same fixed-budget output. Empirically, most discovered-but-unselected correct answers lose reachability only after discovery. In a controlled Word16 study, input permutation improves raw-plurality accuracy by 21.1 points with essentially unchanged single-call correctness. Exact locking saves 28-30% of calls at a 16-call budget while preserving every fixed-budget output.
Sources
- When Self-Consistency Backfires: Majority Vote Hurts the Majority of Hard Science Problems for Small LLMs
- When More Sampling Hurts: The Modal Ceiling and Correlation Ceiling of Test-Time Scaling
- Large Language Monkeys: Scaling Inference Compute with Repeated Sampling
- MARS: Margin-Adversarial Risk-controlled Stopping for Parallel LLM Test-time Scaling
- Training Verifiers to Solve Math Word Problems
- Gemma 4 Technical Report
- The Llama 3 Herd of Models
- Oracle Gap and Signal Fidelity: A Fixed-Pool Diagnostic for Test-Time Collaboration
- Candidate supply and answer selection shape the value of LLM judging in multi-agent systems
- Ministral 3
- EMS: Multi-Agent Voting via Efficient Majority-then-Stopping
- Single-Agent LLMs Outperform Multi-Agent Systems on Multi-Hop Reasoning Under Equal Thinking Token Budgets
- Qwen3 Technical Report
Related papers
- MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
- Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
- Agents' Last Exam
- The Clinician's Veto: Navigating Trust, Liability, and Uncertainty in Autonomous AI Prescribing
- MindHelper: Closed-Loop Embodied Mental-State Reasoning for Precision Intervention
- Incumbent Advantage: Brand Bias and Cognitive Manipulation Dynamics in LLM Recommendation Systems