iMuon: Intrinsic Spectral Optimization on Riemannian Matrix Manifolds
cs.LG, cs.AI
Submitted: 2026-05-10
Updated: 2026-11-16
Code: https://github.com/1bang118/manifold-intrinsic-muon
Terminology
Sources
- The Polar Express: Optimal Matrix Sign Methods and Their Application to the Muon Algorithm
- Old Optimizer, New Norm: An Anthology
- Modular Duality in Deep Learning
- LoRA meets Riemannion: Muon Optimizer for Parametrization-independent Low-Rank Adapters
- An Exploration of Non-Euclidean Gradient Descent: Muon and its Many Variants
- Mano: Restriking Manifold Optimization for LLM Training
- Stabilizing Native Low-Rank LLM Pretraining
- Uniform Spectral Growth and Convergence of Muon in LoRA-Style Matrix Factorization
- Convergence of Muon with Newton-Schulz
- Scalable Optimization in the Modular Norm
- A Note on the Convergence of Muon
- Preconditioning Benefits of Spectral Orthogonalization in Muon
- A Riemannian geometry for low-rank matrix completion
- NuMuon: Nuclear-Norm-Constrained Muon for Compressible LLM Training
- Riemannian Optimization for LoRA on the Stiefel Manifold
- Training Deep Learning Models with Norm-Constrained LMOs
- Gluon: Making Muon & Scion Great Again! (Bridging Theory and Practice of LMO-based Optimizers for LLMs)
- Convergence Bound and Critical Batch Size of Muon Optimizer
- A geometric framework for momentum-based optimizers for low-rank training
- On the Convergence Analysis of Muon
Related papers
- A Physics-Regulated Neural Framework for Learning 3D Grain Growth Dynamics
- FlexMoE: Learning One-for-All Multi-Budget Expert Pruning for Flexible MoE Deployment
- OptiMind: Improving Small Language Models for Optimization Modeling through Formulation Guidance and Multi-Turn Learning
- Staying in the Sweet Spot: Boost Reasoning Evolution via Responsive Rollout Difficulty Manipulation
- Examining the Impact of Data and Training Methods on Emergent and Subliminal Misalignment
- Rethinking Black-Box On-Policy Distillation: A Rubric-based Analysis