LEAD: Length-Efficient Adaptive and Dynamic Reasoning for Large Language Models
cs.LG, cs.AI
Submitted: 2026-05-10
Updated: 2026-11-02
Comments: NeurIPS 2026
Code: https://github.com/CrazyMint/LEAD
License: http://creativecommons.org/licenses/by/4.0/
The gist: Large reasoning models, such as OpenAI o1 and DeepSeek-R1, tend to become increasingly verbose as their reasoning capabilities improve.
Terminology
Abstract
Large reasoning models, such as OpenAI o1 and DeepSeek-R1, tend to become increasingly verbose as their reasoning capabilities improve. These inflated Chain-of-Thought (CoT) trajectories often exceed what the underlying problems require, wasting compute, latency, and context budgets. While introducing length-based efficiency rewards during reinforcement learning offers a natural remedy, existing methods struggle with two fundamental challenges: the optimal balance between correctness and efficiency is non-stationary throughout training, and intrinsic reasoning budgets vary drastically across problems. Relying on static reward weights and global length constraints inevitably forces a compromise between degraded accuracy and unrealized compression. To overcome these limitations, we propose LEAD (Length-Efficient Adaptive and Dynamic reasoning), a method that replaces static heuristics with online, self-adaptive mechanisms. LEAD dynamically calibrates the correctness--efficiency trade-off at each step using a Potential-Scaled Instability, directing optimization capacity to the most informative learning signal. Furthermore, it estimates an adaptive per-problem target length online based on the model's own correct rollouts, applying a symmetric efficiency reward that penalizes both overthinking and over-compression. Evaluated on five mathematical reasoning benchmarks, LEAD achieves the highest accuracy and Accuracy-Efficiency Score among RL-trained efficient-reasoning methods in our matched comparisons while producing substantially shorter outputs than the base model.
Sources
- Self-Consistency Improves Chain of Thought Reasoning in Language Models
- Least-to-Most Prompting Enables Complex Reasoning in Large Language Models
- ReAct: Synergizing Reasoning and Acting in Language Models
- OpenAI o1 System Card
- DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning
- Do NOT Think That Much for 2+3=? On the Overthinking of o1-Like LLMs
- TokenFlow: Responsive LLM Text Streaming Serving under Request Burst via Preemptive Scheduling
- Training Language Models to Reason Efficiently
- Just Enough Thinking: Efficient Reasoning with Adaptive Length Penalties Reinforcement Learning
- L1: Controlling How Long A Reasoning Model Thinks With Reinforcement Learning
- O1-Pruner: Length-Harmonizing Fine-Tuning for O1-Like Reasoning Pruning
- ShorterBetter: Guiding Reasoning Models to Find Optimal Inference Length for Efficient Reasoning
- SmartThinker: Learning High-Density Reasoning via Step-Level Token Allocation
- DRPO: Efficient Reasoning via Decoupled Reward Policy Optimization
- Learn to Reason Efficiently with Adaptive Length-based Reward Shaping
- SelfBudgeter: Adaptive Token Allocation for Efficient LLM Reasoning
- Sample More to Think Less: Group Filtered Policy Optimization for Concise Reasoning
- Kimi k1.5: Scaling Reinforcement Learning with LLMs
- HAPO: Training Language Models to Reason Concisely via History-Aware Policy Optimization
- GDPO: Group reward-Decoupled Normalization Policy Optimization for Multi-reward RL Optimization
Related papers
- A Physics-Regulated Neural Framework for Learning 3D Grain Growth Dynamics
- FlexMoE: Learning One-for-All Multi-Budget Expert Pruning for Flexible MoE Deployment
- OptiMind: Improving Small Language Models for Optimization Modeling through Formulation Guidance and Multi-Turn Learning
- Staying in the Sweet Spot: Boost Reasoning Evolution via Responsive Rollout Difficulty Manipulation
- Examining the Impact of Data and Training Methods on Emergent and Subliminal Misalignment
- Rethinking Black-Box On-Policy Distillation: A Rubric-based Analysis