Byzantine-Robust Muon: Resilient Momentum Orthogonalization for Distributed Learning
cs.LG
Submitted: 2026-11-09
Updated: 2026-11-09
Terminology
Sources
- GPT-4 Technical Report
- Dion: Distributed Orthonormalized Updates
- The Polar Express: Optimal Matrix Sign Methods and Their Application to the Muon Algorithm
- ASGO: Adaptive Structured Gradient Optimization
- Old Optimizer, New Norm: An Anthology
- Byzantine-Robust Optimization under $(L_0, L_1)$-Smoothness
- An Exploration of Non-Euclidean Gradient Descent: Muon and its Many Variants
- Distance-Aware Muon: Adaptive Step Scaling for Normalized Optimization
- TinyStories: How Small Can Language Models Be and Still Speak Coherent English?
- The Llama 3 Herd of Models
- Non-Euclidean Broximal Point Method: A Blueprint for Geometry-Aware Optimization
- Error Feedback for Muon and Friends
- Drop-Muon: Update Less, Converge Faster
- A Unified Approach to Adaptive Regularization in Online and Stochastic Optimization
- Secure Byzantine-Robust Machine Learning
- Measuring the Effects of Non-Identical Data Distribution for Federated Visual Classification
- Byzantine-Robust and Differentially Private Federated Optimization under Weaker Assumptions
- MuonBP: Faster Muon via Block-Periodic Orthogonalization
- Better LMO-based Momentum Methods with Second-Order Information
- Understanding Gradient Orthogonalization for Deep Learning via Non-Euclidean Trust-Region Optimization
Related papers
- A Physics-Regulated Neural Framework for Learning 3D Grain Growth Dynamics
- FlexMoE: Learning One-for-All Multi-Budget Expert Pruning for Flexible MoE Deployment
- OptiMind: Improving Small Language Models for Optimization Modeling through Formulation Guidance and Multi-Turn Learning
- Staying in the Sweet Spot: Boost Reasoning Evolution via Responsive Rollout Difficulty Manipulation
- Examining the Impact of Data and Training Methods on Emergent and Subliminal Misalignment
- Rethinking Black-Box On-Policy Distillation: A Rubric-based Analysis