Improving Sampling Efficiency in RLVR through Adaptive Rollout and Response Reuse
cs.LG
Submitted: 2025-09-30
Updated: 2026-11-01
Code: https://github.com/huggingface/Math-Verify
Terminology
Sources
- GPT-4 Technical Report
- Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback
- Evaluating Large Language Models Trained on Code
- RLHF Workflow: From Reward Modeling to Online RLHF
- KTO: Model Alignment as Prospect Theoretic Optimization
- DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning
- OlympiadBench: A Challenging Benchmark for Promoting AGI with Olympiad-Level Bilingual Multimodal Scientific Problems
- Measuring Mathematical Problem Solving With the MATH Dataset
- ORPO: Monolithic Preference Optimization without Reference Model
- OpenAI o1 System Card
- Enhancing Efficiency and Exploration in Reinforcement Learning for LLMs
- Understanding R1-Zero-Like Training: A Critical Perspective
- Nash Learning from Human Feedback
- Qwen2 Technical Report
- Proximal Policy Optimization Algorithms
- DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models
- Improving Data Efficiency for LLM Reinforcement Fine-tuning Through Difficulty-targeted Online Data Selection and Rollout Replay
- Kimi k1.5: Scaling Reinforcement Learning with LLMs
- Self-Play Preference Optimization for Language Model Alignment
- Exploratory Preference Optimization: Harnessing Implicit Q*-Approximation for Sample-Efficient RLHF
Related papers
- A Physics-Regulated Neural Framework for Learning 3D Grain Growth Dynamics
- FlexMoE: Learning One-for-All Multi-Budget Expert Pruning for Flexible MoE Deployment
- OptiMind: Improving Small Language Models for Optimization Modeling through Formulation Guidance and Multi-Turn Learning
- Staying in the Sweet Spot: Boost Reasoning Evolution via Responsive Rollout Difficulty Manipulation
- Examining the Impact of Data and Training Methods on Emergent and Subliminal Misalignment
- Rethinking Black-Box On-Policy Distillation: A Rubric-based Analysis