PRISM: Pushing the Frontier of Deep Think via Process Reward Model-Guided Inference
cs.AI
Submitted: 2026-03-03
Updated: 2026-10-29
Comments: 28 pages, 22 figures, 4 tables
Code: https://github.com/Rituraj003/PRISM
License: http://creativecommons.org/licenses/by/4.0/
The gist: DEEPTHINK methods improve reasoning by generating, refining, and aggregating populations of candidate solutions, which enables strong performance on complex mathematical and scientific tasks.
Terminology
Abstract
DEEPTHINK methods improve reasoning by generating, refining, and aggregating populations of candidate solutions, which enables strong performance on complex mathematical and scientific tasks. However, existing frameworks can still suffer from unreliable correctness signals during inference, which creates a population- enhancement bottleneck where deeper deliberation amplifies errors, suppresses correct minority solutions, and yields weak returns to additional compute. In this paper, we introduce a functional decomposition of DEEPTHINK systems and propose PRISM, a Process Reward Model (PRM)-guided inference algorithm that uses step-level verification to guide resampling, proposal acceptance, and solution aggregation. During refinement, PRISM uses PRM scores to guide population resampling and stochastic local refinement, which concentrates computation on higher-quality reasoning while preserving diversity. When paired with PRM- score-vote, a PRM-guided aggregation method, PRISM is competitive with or outperforms existing DEEPTHINK methods, reaching 90.0%, 75.4%, and 71.4% with gpt-oss-20b on AIME25, HMMT25, and GPQA Diamond, respectively, while exceeding gpt-oss-120b. Additionally, our analysis shows that PRISM produces consistent net-directional correction during refinement and maintains competitive accuracy when the initial population contains few correct candidates.
Sources
- gpt-oss-120b & gpt-oss-20b Model Card
- SciMaster: Towards General-Purpose Scientific AI Agents, Part I. X-Master as Foundation: Can We Lead on Humanity's Last Exam?
- Recursive Self-Aggregation Unlocks Deep Thinking in Large Language Models
- Qwen3 Technical Report
- Verbalized Sampling: How to Mitigate Mode Collapse and Unlock LLM Diversity
Related papers
- Multi-Objective Structured Pruning of LLMs for Latency and Model Size Optimization
- MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
- Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
- Embeddings for Preferences, Not Semantics
- Can We Trust Item Response Theory for AI Evaluation?
- Diagnosing Task Insensitivity in Language Agents