Learning to Solve, Forgetting to Retain: Correct-Set Turnover in RLVR

arXiv:2606.03087 · cs.LG · Submitted 2026-06-02 · Read on arXiv

cs.LG

Submitted: 2026-06-02

Updated: 2026-09-29

Code: https://github.com/cyuQ1n/Correct-Set-Turnover-in-RLVR

Terminology

• Accuracy — 1× in this paper · explained in 1 episode
• Benchmark — 1× in this paper · explained in 2 episodes
• GRPO — 1× in this paper · explained in 6 episodes
• Generalizability — 1× in this paper · explained in 6 episodes
• Reinforcement Learning — 1× in this paper · explained in 4 episodes
• Verifiable Rewards — 1× in this paper · explained in 2 episodes

Sources

• Online Difficulty Filtering for Reasoning Oriented Reinforcement Learning
• Qwen3-VL Technical Report
• Scaling RL to Long Videos
• Video-Holmes: Can MLLM Think Like Holmes for Complex Video Reasoning?
• The Entropy Mechanism of Reinforcement Learning for Reasoning Language Models
• Probing RLVR training instability through the lens of objective-level hacking
• Improving RL Exploration for LLM Reasoning through Retrospective Replay
• DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning
• Measuring Mathematical Problem Solving With the MATH Dataset
• MotionBench: Benchmarking and Improving Fine-grained Video Motion Understanding for Vision Language Models
• Emergent Slow Thinking in LLMs as Inverse Tree Freezing
• VCRL: Variance-based Curriculum Reinforcement Learning for Large Language Models
• MVBench: A Comprehensive Multi-modal Video Understanding Benchmark
• RePO: Replay-Enhanced Policy Optimization
• MCPO: Mastery-Consolidated Policy Optimization for Large Reasoning Models
• MMFineReason: Closing the Multimodal Reasoning Gap via Open Data-Centric Methods
• Understanding R1-Zero-Like Training: A Critical Perspective
• Freshness-Aware Prioritized Experience Replay for LLM/VLM Reinforcement Learning
• Beyond Reasoning Gains: Mitigating General-Capability Forgetting in Large Reasoning Models
• VideoMathQA: Benchmarking Mathematical Reasoning via Multimodal Understanding in Videos

Related papers

• Polynomial-Augmented Neural Networks (PANNs) with Weak Orthogonality Constraints for Enhanced Function and PDE Approximation
• AIRL-S: Unifying Reinforcement Learning and Search-Based Test-Time Scaling via Adversarial Inverse Reinforcement Learning
• Transformers as Bayesian In-Context Experimenters: Smoothness-Adaptive Efficient ATE Estimation
• Convergence issues in Relational Concept Analysis based on AOC-posets
• Beliefs Beyond Posteriors: The Predictive Cost of ELBO Binding
• Understanding Diffusion Models via Ratio-Based Function Approximation with SignReLU Networks