DMuon-2: Scaling Matrix Optimizers in Data Parallelism with Topology-Aware Matrix Ownership and Communication
cs.DC
Submitted: 2026-11-04
Updated: 2026-11-06
Code: https://github.com/X-Square-Robot/dmu
Terminology
Sources
- Dion: Distributed Orthonormalized Updates
- The Polar Express: Optimal Matrix Sign Methods and Their Application to the Muon Algorithm
- DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligence
- MatrixFSDP: communication-free matrix optimizers under ZeRO-3 parameter sharding
- GLM-5: from Vibe Coding to Agentic Engineering
- GPU-Initiated Networking for NCCL
- HiCCL: A Hierarchical Collective Communication Library
- Kimi K3: Open Frontier Intelligence
- WALL-WM: Carving World Action Modeling at the Event Joints
- Motif 2 12.7B technical report
- LIBERO: Benchmarking Knowledge Transfer for Lifelong Robot Learning
- Muon is Scalable for LLM Training
- The FineWeb Datasets: Decanting the Web for the Finest Text Data at Scale
- $\pi_{0.5}$: a Vision-Language-Action Model with Open-World Generalization
- A Distributed Data-Parallel PyTorch Implementation of the Distributed Shampoo Optimizer for Training Neural Networks At-Scale
- Step 3.5 Flash: Open Frontier-Level Intelligence with 11B Active Parameters
- Canzona: A Unified, Asynchronous, and Load-Balanced Framework for Distributed Matrix-based Optimizers
- veScale-FSDP: Flexible and High-Performance FSDP at Scale
- Qwen2.5 Technical Report
- Wall-OSS-0.5 Technical Report
Related papers
- Programmable Probabilistic Computer with 1,000,000 p-bits
- BatchGen: An Architecture for Scalable and Efficient Batch Inference
- Seamless Reconfiguration for DAG-BFT (Technical Report)
- ELDR: Expert-Locality-Aware Decode Routing for PD-Disaggregated MoE Serving
- iScheduler: Reinforcement Learning-Driven Continual Optimization for Large-Scale Resource Investment Problems
- TIRx: A Unified Tile Primitive ML Compiler Across Heterogeneous Backends