Optimizing Anytime Reasoning via Budget Relative Policy Optimization
cs.LG, cs.AI, cs.CL
Submitted: 2025-05-19
Updated: 2026-10-30
Code: https://github.com/sail-sg/AnytimeReasoner
License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
Terminology
Sources
- L1: Controlling How Long A Reasoning Model Thinks With Reinforcement Learning
- Back to Basics: Revisiting REINFORCE Style Optimization for Learning from Human Feedback in LLMs
- Training Language Models to Reason Efficiently
- A Tutorial on Meta-Reinforcement Learning
- Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilities
- Process Reinforcement through Implicit Rewards
- The Entropy Mechanism of Reinforcement Learning for Reasoning Language Models
- DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning
- Token-Budget-Aware LLM Reasoning
- OlympiadBench: A Challenging Benchmark for Promoting AGI with Olympiad-Level Bilingual Multimodal Scientific Problems
- Measuring Mathematical Problem Solving With the MATH Dataset
- VinePPO: Refining Credit Assignment in RL Training of LLMs
- Tulu 3: Pushing Frontiers in Open Language Model Post-Training
- How Well do LLMs Compress Their Own Chain-of-Thought? A Token Complexity Approach
- ReMax: A Simple, Effective, and Efficient Reinforcement Learning Method for Aligning Large Language Models
- Understanding R1-Zero-Like Training: A Critical Perspective
- Reasoning Models Can Be Effective Without Thinking
- s1: Simple test-time scaling
- Concise Thoughts: Impact of Output Length on LLM Reasoning and Cost
- Optimizing Test-Time Compute via Meta Reinforcement Fine-Tuning
Related papers
- A Physics-Regulated Neural Framework for Learning 3D Grain Growth Dynamics
- FlexMoE: Learning One-for-All Multi-Budget Expert Pruning for Flexible MoE Deployment
- OptiMind: Improving Small Language Models for Optimization Modeling through Formulation Guidance and Multi-Turn Learning
- Staying in the Sweet Spot: Boost Reasoning Evolution via Responsive Rollout Difficulty Manipulation
- Examining the Impact of Data and Training Methods on Emergent and Subliminal Misalignment
- Rethinking Black-Box On-Policy Distillation: A Rubric-based Analysis