Consilience for Verifier-Free Test-Time Scaling

The paper addresses verifier-free test-time scaling (VF-TTS) for Large Language Models (LLMs), which is gaining attention because many real-world applications lack access to high-quality external verifiers (such as compilers, test cases, or trained value functions). Among existing VF-TTS methods, confidence-based approaches—which compute and rank rollouts solely by confidence—are particularly promising because they introduce near-zero overhead for sample evaluation and require minimal access to internal model states.

The authors demonstrate a critical limitation of existing confidence-based VF-TTS methods: they catastrophically break down on complex tasks. They observe that uniformly high confidence frequently indicates a failure to explore, favoring confidently wrong answers.

The paper identifies a phenomenon where confidence maximization fails on hard problems. While on a macro level, higher average confidence correlates with correctness (correct answers have mean confidence µ = 10.83 vs. incorrect µ = 9.12 on LiveCodeBench-V6 using GPT-OSS-120B), this pattern reverses on difficult problems (correct rate < 20%): incorrect answers exhibit higher mean confidence (µ = 9.03) than correct ones (µ = 8.79). The incorrect distribution becomes highly over-dispersed, forming a heavy 'long tail' at the highest confidence percentiles.

The core insight is that robust cognitive search requires a specific confidence trajectory pattern: methods should perform exploratory branching at the beginning, as manifested by low initial confidence, and converge to a high final confidence solution. This aligns with the principle of consilience—a conclusion is reliable only if the model knows diverse paths to solve the problem and all of the paths converge to the same solution.

The authors introduce consilience, a novel selection framework that explicitly evaluates the temporal asymmetry of confidence in reasoning. The method:

1. Defines token-level confidence as the negative mean log-probability over the top-K candidate tokens: cy,t = −(1/K)Σ log Pθ(vt,jy<t, x)

2. Identifies four categories of reasoning trajectories:

• Category 1 (Premature Convergence): High initial + High final confidence—model collapses onto a singular path, leading to confidently wrong answers

• Category 2 (Degeneration): High initial + Low final confidence—model confidently begins but encounters contradictions

• Category 3 (Consilience): Low initial + High final confidence—model sees diverse paths before converging (the optimal category)

• Category 4 (Total Confusion): Low initial + Low final confidence—prompt too difficult

3. Formulates the Consilience Score: S = C final − α·C initial, where C initial and C final are average token confidence over the first and last W tokens (with the first P tokens skipped), and α ≥ 0 is a balancing factor. This actively penalizes high initial confidence while strictly demanding final certainty.

4. Implements reasoning-phase isolation: The method parses the sequence to identify the semantic transition boundary (e.g., for DeepSeek-R1, final for GPT-OSS) and only evaluates the reasoning portion, removing the artificially confident summarization tokens that dilute the signal.

The paper evaluates across multiple models (GPT-OSS-120B/20B, DeepSeek-R1-Qwen7B-Distill, Qwen3-80B-Next-Instruct-FP8) and datasets (LiveCodeBench-v6, HMMT25, GPQA-Diamond, SWE-bench):

LiveCodeBench (code generation):

• GPT-OSS-120B: Pass@1 = 65.7%, Consilience-think = 69.7% (vs. Self-Certainty = 64.5%, DeepConf-20% = 61.9%)

• Qwen: Pass@1 = 55.3%, Consilience-think = 60.4% (vs. Self-Certainty = 53.2%)

HMMT (mathematics):

• GPT-OSS-20B: Pass@1 = 76.5%, Consilience-think = 78.0% (vs. Self-Certainty = 68.0%)

• Qwen: Pass@1 = 52.6%, Consilience-think = 52.0%

GPQA (graduate-level QA):

• GPT-OSS-20B: Pass@1 = 71.8%, Consilience-think = 72.0%

• Qwen: Pass@1 = 60.5%, Consilience-full = 65.2%

SWE-bench (agentic coding):

• GPT-OSS-120B: 23.0% → 26.9% with consilience

• Qwen3-Coder-Next: 65.3% → 67.3% with consilience

1. Failure of simple maximization: Applying these metrics without voting frequently results in catastrophic performance degradation compared to random selection, Pass@1. For example, Self-Certainty drops HMMT performance from 76.5% to 68.0% on OSS-20B.

2. Scaling robustness: Consilience generally exhibits superior and more consistent scaling properties as sample size increases to 128. On GPQA, consilience scales at a visibly steeper rate. On LCB with Qwen, only consilience scales positively.

3. Temporal confidence trajectories: On hard problems (Pass@1 < 50%), the difference between correct and incorrect completions' confidence is negative or lowest at the beginning of the sequence and steadily rises toward the end, confirming that correct reasoning has lower initial confidence.

4. Statistical significance: On hard problems (Pass@1 < 20%), consilience separates correct from incorrect completions with AUROC 0.61–0.62 (non-overlapping 95% CIs), whereas mean confidence is at chance (0.47–0.50).

5. Difficulty stratification: Consilience is neutral on easy problems and improves on medium/hard tiers. On medium problems, gains reach +13.7 points (Qwen on LCB: 58.1% → 70.3%).

6. Hyperparameter robustness: The initial-confidence penalty consistently yields a positive performance boost across a massive combinatorial range of percentages. A generic robust range is 20% to 30% for window sizes, and α ∈ [2, 5] for the penalty multiplier. Frozen parameters (α = 3, k = 20%) capture 72–78% of the per-set optimal gain and transfer across datasets.

The paper provides a case study on a hard LiveCodeBench problem (pass@1 = 0.03) showing that the consilience pick (correct) spent more time on critical design choice and questioned its initial framing, while the mean-confidence pick (incorrect) accepts this framing immediately and commits to code without revisiting a flawed stop-on-duplicate assumption. The consilience completion identified the key insight: we cannot have any path from any ancestor before the previous occurrence of nums[u] to u... but we can have paths from ancestors after the previous occurrence... Wait, that's key!

The paper positions consilience as an instance of a general test-time-scaling principle: that exploration should be valued rather than penalized. It is distinct from prior work because:

1. It operationalizes the temporal confidence pattern as an active selection objective rather than just observing it or using it to reweight votes

2. It is training-free and computed from logits alone, applicable to free-form generation where voting is inapplicable

3. It uniquely isolates the reasoning phase from the final-answer phase before scoring

Future directions include learned or judge-based selectors, extension to tree-structured search, smarter agentic harnesses for efficiency, and using consilience as an intrinsic reward for agentic reinforcement learning.

Based on the paper, here are the specific improvements I can make to AI systems:

Improvement: Implement a temporal confidence asymmetry scoring mechanism that penalizes high initial confidence and rewards high final confidence during multi-sample generation.

What the improved system can do:

• Automatically identify and discard premature convergence trajectories where the model commits to a single reasoning path too early

• Select solutions that demonstrate genuine exploration followed by convergence, rather than those that are simply confident throughout

• Reduce confidently-wrong answers on complex tasks by up to 8.2 percentage points (e.g., 65.7% → 69.7% on LiveCodeBench with GPT-OSS-120B)

Improvement: Parse model outputs to identify semantic boundaries (e.g., ``, final) and evaluate confidence only on the reasoning portion, excluding artificially confident summarization tokens.

Improvement: Implement a two-mode selection system: use simple mean confidence for easy problems, and switch to consilience scoring for medium/hard problems (identified via low pass@1 rates or low initial confidence).

Improvement: Use the initial-confidence signal to guide sampling—when initial confidence is low, encourage more diverse branching; when initial confidence is high, reduce sample diversity and focus on refinement.

Improvement: Deploy consilience as a lightweight, logits-only selection criterion that requires no external verifiers, trained value functions, or voting mechanisms.

Improvement: Use the consilience score as an intrinsic reward component for agentic RL training, rewarding trajectories that show exploratory branching followed by confident convergence.

Improvement: Monitor temporal confidence trajectories during generation to detect premature convergence or degeneration patterns, enabling early termination or re-generation.

Test-time scaling often uses an external verifier, such as compilers and test cases in coding or trained value functions in robotics applications, to obtain high-quality rollouts. Verifier-free test-time scaling (or VF-TTS) is gaining extensive attention as a mechanism to enhance Large Language Model (LLM) reasoning, primarily because we do not have access to such high-quality verifiers in many real-world applications. Among existing VF-TTS methods, confidence-based VF-TTS methods, which compute and rank rollouts solely by confidence, are particularly promising. Such methods introduce near-zero overhead for sample evaluation and require minimal access to internal model states, making the methods highly flexible across models and tasks. In this paper, we demonstrate a critical limitation of existing confidence-based VF-TTS methods by showing that such methods catastrophically break down on complex tasks. We observe a very interesting phenomenon: uniformly high confidence frequently indicates a failure to explore, favoring confidently wrong answers. To address this, our core insight is that robust cognitive search requires a specific confidence trajectory pattern: such methods perform exploratory branching at the beginning, as manifested by low initial confidence, and converge to a high final confidence solution. To implement this insight, we introduce consilience, a novel selection framework that explicitly evaluates the temporal asymmetry of confidence in reasoning. We operationalize this via a combinatorial metric that actively penalizes high initial confidence while strictly demanding final certainty. Extensive experiments covering both graduate-level mathematics problems and free-form code generation demonstrate that consilience effectively outperforms existing baselines, validating our novel perspective on completion confidence.