Decoupling Exploration from Optimization in RLVR

arXiv:2610.10536 · cs.LG, cs.AI, cs.CL · Submitted 2026-10-07 · Read on arXiv

cs.LG, cs.AI, cs.CL

Submitted: 2026-10-07

Updated: 2026-10-07

Terminology

• Decoupling — 1× in this paper · explained in 2 episodes

Sources

• Exploration by Random Network Distillation
• Evaluating Large Language Models Trained on Code
• Training Verifiers to Solve Math Word Problems
• The Entropy Mechanism of Reinforcement Learning for Reasoning Language Models
• Instruction-Conditioned Exploration for Reinforcement Learning with Self-Distillation to an Unconditioned Policy
• Diversity is All You Need: Learning Skills without a Reward Function
• Navigate the Unknown: Enhancing LLM Reasoning with Intrinsic Motivation Guided Exploration
• Reinforced Self-Training (ReST) for Language Modeling
• DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning
• Measuring Mathematical Problem Solving With the MATH Dataset
• Tulu 3: Pushing Frontiers in Open Language Model Post-Training
• Jointly Reinforcing Diversity and Quality in Language Model Generations
• Ministral 3
• MOPD: Multi-Teacher On-Policy Distillation for Capability Integration in LLM Post-Training
• When Can LLMs Learn to Reason with Weak Supervision?
• GPQA: A Graduate-Level Google-Proof Q&A Benchmark
• DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models
• Beyond Human Data: Scaling Self-Training for Problem-Solving with Language Models
• Outcome-based Exploration for LLM Reasoning
• R$^2$PO: Decoupling Rollout and Inference Policies for LLM Reasoning

Related papers

• Polynomial-Augmented Neural Networks (PANNs) with Weak Orthogonality Constraints for Enhanced Function and PDE Approximation
• AIRL-S: Unifying Reinforcement Learning and Search-Based Test-Time Scaling via Adversarial Inverse Reinforcement Learning
• Transformers as Bayesian In-Context Experimenters: Smoothness-Adaptive Efficient ATE Estimation
• Convergence issues in Relational Concept Analysis based on AOC-posets
• Beliefs Beyond Posteriors: The Predictive Cost of ELBO Binding
• Understanding Diffusion Models via Ratio-Based Function Approximation with SignReLU Networks