Stabilizing MoE Reinforcement Learning by Aligning Training and Inference Routers
cs.CL, cs.AI, cs.LG
Submitted: 2025-10-13
Updated: 2026-11-09
Journal ref: Proceedings of the 43rd International Conference on Machine Learning, Proceedings of Machine Learning Research 306:85320-85343, 2026
Code: https://github.com/huggingface/open-r1
License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
Terminology
Sources
- Big-Math: A Large-Scale, High-Quality Math Dataset for Reinforcement Learning in Language Models
- Llama-Nemotron: Efficient Reasoning Models
- MiniMax-M1: Scaling Test-Time Compute Efficiently with Lightning Attention
- StableMoE: Stable Routing Strategy for Mixture of Experts
- DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning
- On-Policy RL with Optimal Reward Baseline
- Open-Reasoner-Zero: An Open Source Approach to Scaling Up Reinforcement Learning on the Base Model
- R2E-Gym: Procedural Environments and Hybrid Verifiers for Scaling Open-Weights SWE Agents
- AM-Thinking-v1: Advancing the Frontier of Reasoning at 32B Scale
- Mixtral of Experts
- SWE-bench: Can Language Models Resolve Real-World GitHub Issues?
- DeepSeek-V3 Technical Report
- Proximal Policy Optimization Algorithms
- DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models
- HybridFlow: A Flexible and Efficient RLHF Framework
- Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism
- Kimi k1.5: Scaling Reinforcement Learning with LLMs
- Continual Pre-training of MoEs: How robust is your router?
- BrowseComp: A Simple Yet Challenging Benchmark for Browsing Agents
- MiMo: Unlocking the Reasoning Potential of Language Model -- From Pretraining to Posttraining
Related papers
- ALARM: Audio-Language Alignment for Reasoning Models
- ThinkSum: Probabilistic reasoning over sets using large language models
- Coherence boosting: When your pretrained language model is not paying enough attention
- From Facts to Insights: A Persona-Driven Dual Memory Framework and Dataset for Role-Playing Agents
- Improving General Role-Playing Agents via Psychology-Grounded Reasoning and Role-Aware Policy Optimization
- The Art of Asking: Multilingual Prompt Optimization for Synthetic Data