HSRM: Hidden-State Reward Models for Test-Time Verification

arXiv:2608.30841 · cs.AI, cs.CL · Submitted 2026-08-31 · Read on arXiv

cs.AI, cs.CL

Submitted: 2026-08-31

Updated: 2026-08-31

Comments: EMNLP 2026

Code: https://github.com/JXL884/HSRM

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Terminology

• Verifier — 93× in this paper · explained in 2 episodes
• Verification — 40× in this paper · explained in 2 episodes
• Accuracy — 36× in this paper · explained in 1 episode
• Transformer — 25× in this paper · explained in 2 episodes
• Ablation — 14× in this paper · explained in 1 episode
• Benchmark — 11× in this paper · explained in 2 episodes
• Trace — 11× in this paper · explained in 2 episodes
• Thinking mode — 5× in this paper · explained in 2 episodes
• Zero-Shot Transfer — 3× in this paper · explained in 4 episodes
• LLM judge — 2× in this paper · explained in 2 episodes
• Ablation Studies — 1× in this paper · explained in 1 episode
• Abstraction — 1× in this paper · explained in 1 episode
• Adaptation — 1× in this paper · explained in 1 episode
• Best-of-N Sampling — 1× in this paper · explained in 2 episodes
• Clustering — 1× in this paper · explained in 2 episodes
• Consistency — 1× in this paper · explained in 3 episodes
• Distribution Shift — 1× in this paper · explained in 5 episodes
• Generalization — 1× in this paper · explained in 13 episodes
• Hallucination — 1× in this paper · explained in 3 episodes
• Large Language Models (LLMs) — 1× in this paper · explained in 6 episodes
• Loss Function — 1× in this paper · explained in 2 episodes
• Parameter Efficiency — 1× in this paper · explained in 3 episodes
• Reasoning Traces — 1× in this paper · explained in 3 episodes
• Representation Engineering — 1× in this paper · explained in 2 episodes
• Scaling Laws — 1× in this paper · explained in 4 episodes
• Self-consistency — 1× in this paper · explained in 2 episodes
• Test-time Scaling — 1× in this paper · explained in 2 episodes
• Transferability — 1× in this paper · explained in 4 episodes

Sources

• Measuring Mathematical Problem Solving With the MATH Dataset
• Learning to Rank Chain-of-Thought: Using a Small Model
• Language Models (Mostly) Know What They Know
• Large Language Monkeys: Scaling Inference Compute with Repeated Sampling
• CLUE: Non-parametric Verification from Experience via Hidden-State Clustering
• Discovering Latent Knowledge in Language Models Without Supervision
• Training Verifiers to Solve Math Word Problems
• The Llama 3 Herd of Models
• Mining Intrinsic Rewards from LLM Hidden States for Efficient Best-of-N Sampling
• Let's Verify Step by Step
• The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasets
• ReProbe: Efficient Test-Time Scaling of Multi-Step Reasoning by Probing Internal States of Large Language Models
• Qwen3 Technical Report
• Reasoning Models Know When They're Right: Probing Hidden States for Self-Verification
• Lightweight Latent Verifiers for Efficient Meta-Generation Strategies
• Layer by Layer: Uncovering Hidden Representations in Language Models
• Solving math word problems with process- and outcome-based feedback
• Inference Scaling Laws: An Empirical Analysis of Compute-Optimal Inference for Problem-Solving with Language Models
• Qwen2.5-Math Technical Report: Toward Mathematical Expert Model via Self-Improvement
• ProcessBench: Identifying Process Errors in Mathematical Reasoning

Related papers

• MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
• Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
• Agents' Last Exam
• The Clinician's Veto: Navigating Trust, Liability, and Uncertainty in Autonomous AI Prescribing
• MindHelper: Closed-Loop Embodied Mental-State Reasoning for Precision Intervention
• Incumbent Advantage: Brand Bias and Cognitive Manipulation Dynamics in LLM Recommendation Systems