Closing the BF16-FP16 Convergence Speed Gap in RL Post-Training with Stochastic Rounding
cs.LG, stat.ML
Submitted: 2026-11-10
Updated: 2026-11-10
Terminology
Sources
- DeepSeek LLM: Scaling Open-Source Language Models with Longtermism
- DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning
- DeepSeek-V3.2: Pushing the Frontier of Open Large Language Models
- A Study of BFLOAT16 for Deep Learning Training
- Understanding R1-Zero-Like Training: A Critical Perspective
- The Llama 3 Herd of Models
- Defeating the Training-Inference Mismatch via FP16
- Qwen3 Technical Report
- RL's Razor: Why Online Reinforcement Learning Forgets Less
Related papers
- A Physics-Regulated Neural Framework for Learning 3D Grain Growth Dynamics
- FlexMoE: Learning One-for-All Multi-Budget Expert Pruning for Flexible MoE Deployment
- OptiMind: Improving Small Language Models for Optimization Modeling through Formulation Guidance and Multi-Turn Learning
- Staying in the Sweet Spot: Boost Reasoning Evolution via Responsive Rollout Difficulty Manipulation
- Examining the Impact of Data and Training Methods on Emergent and Subliminal Misalignment
- Rethinking Black-Box On-Policy Distillation: A Rubric-based Analysis