Refining Over Resampling: Test-Time Self-Correction for LLM Reasoning
Ahsan Bilal, Muhammad Ahmed Mohsin, Muhammad Umer, Lena Trigg, Ali Subhan, Muhammad Ali, Dean F. Hougen
cs.AI, cs.CL
Submitted: 2026-08-06
Comments: Submitted to EMNLP 2026
License: http://creativecommons.org/licenses/by/4.0/
The gist: Test-time scaling improves LLM reasoning by using additional inference compute, but wider sampling alone can suffer from diminishing returns: new rollouts often repeat existing answer patterns
Terminology
Abstract
Test-time scaling improves LLM reasoning by using additional inference compute, but wider sampling alone can suffer from diminishing returns: new rollouts often repeat existing answer patterns instead of adding useful reasoning diversity. Verifier-based selection offers an alternative, but its performance depends on the calibration of an external reward model. We propose a verifier-free breadth--depth refinement framework that uses test-time compute to both explore and improve candidate solutions. The method samples multiple independent reasoning rollouts, refines each rollout through iterative self-critique and self-correction, and aggregates the refined answers by majority voting. Breadth preserves diverse initial attempts, while depth repairs local reasoning errors before aggregation. Across AIME24, AIME25, AMC, OlympiadBench, and MATH500, our method consistently improves over greedy decoding, majority voting, verifier-based best-of- N, beam search, and lookahead decoding across multiple open-weight models. For instance, with Qwen2.5-1.5B, accuracy increases from the strongest verifier-based baseline to 58.0% on MATH500, and from 25.0% to 32.5% on AMC. These results show that test-time compute can be more effective when used to refine sampled trajectories rather than only to sample more candidates or rely on verifier-guided selection.
Sources
- OlympiadBench: A Challenging Benchmark for Promoting AGI with Olympiad-Level Bilingual Multimodal Scientific Problems
- Measuring Mathematical Problem Solving With the MATH Dataset
- Training Compute-Optimal Large Language Models
- Theoretical guarantees on the best-of-n alignment policy
- MindStar: Enhancing Math Reasoning in Pre-trained LLMs at Inference Time
- Training Verifiers to Solve Math Word Problems
- Scaling Laws for Neural Language Models
- ROC-n-reroll: How verifier imperfection affects test-time scaling
- Training Language Models to Self-Correct via Reinforcement Learning
- ReVISE: Learning to Refine at Test-Time via Intrinsic Self-Verification
- The Llama 3 Herd of Models
- Reasoning-as-Logic-Units: Scaling Test-Time Reasoning in Large Language Models Through Logic Unit Alignment
- Ministral 3
- Large Language Models have Intrinsic Self-Correction Ability
- Self-Consistency Improves Chain of Thought Reasoning in Language Models
- Self-Refine: Iterative Refinement with Self-Feedback
- Generating Sequences by Learning to Self-Correct
- A Human-Like Reasoning Framework for Multi-Phases Planning Task with Large Language Models
- Qwen2 Technical Report
- Qwen2.5-Math Technical Report: Toward Mathematical Expert Model via Self-Improvement
Related papers
- MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
- Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
- Agents' Last Exam
- The Clinician's Veto: Navigating Trust, Liability, and Uncertainty in Autonomous AI Prescribing
- MindHelper: Closed-Loop Embodied Mental-State Reasoning for Precision Intervention
- Incumbent Advantage: Brand Bias and Cognitive Manipulation Dynamics in LLM Recommendation Systems