Train High, Ship Low: Efficient LoRA Synchronization for Distributed On-Policy Post-Training
cs.LG, cs.AI
Submitted: 2026-11-09
Updated: 2026-11-09
Terminology
Sources
- DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning
- DiLoCo: Distributed Low-Communication Training of Language Models
- Stream of Search (SoS): Learning to Search in Language
- SparseRL-Sync: Lossless Weight Synchronization with ~100x Less Communication
- A Rank Stabilization Scaling Factor for Fine-Tuning with LoRA
- Sequential Beats Joint: On the Interplay between On-Policy Distillation and RLVR
- INTELLECT-2: A Reasoning Model Trained Through Globally Decentralized Reinforcement Learning
- Qwen3 Technical Report
- DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models
- ECHO-2: A Large-Scale Distributed Rollout Framework for Cost-Efficient Reinforcement Learning
- REASONING GYM: Reasoning Environments for Reinforcement Learning with Verifiable Rewards
- On Memorization of Large Language Models in Logical Reasoning
- LoGRA: Scaling LLM Reinforcement Learning with Low-Rank Gradient Sketches
- Self-Distilled Reasoner: On-Policy Self-Distillation for Large Language Models
Related papers
- A Physics-Regulated Neural Framework for Learning 3D Grain Growth Dynamics
- FlexMoE: Learning One-for-All Multi-Budget Expert Pruning for Flexible MoE Deployment
- OptiMind: Improving Small Language Models for Optimization Modeling through Formulation Guidance and Multi-Turn Learning
- Staying in the Sweet Spot: Boost Reasoning Evolution via Responsive Rollout Difficulty Manipulation
- Examining the Impact of Data and Training Methods on Emergent and Subliminal Misalignment
- Rethinking Black-Box On-Policy Distillation: A Rubric-based Analysis