Muon on the Stiefel Manifold Admits an Exact Closed-Form Update

arXiv:2608.06218 · math.OC, cs.LG, math.NA · Submitted 2026-08-06 · Read on arXiv

Mikhail Solonko, Molozhavenko Alexander, Maxim Rakhuba

math.OC, cs.LG, math.NA

Submitted: 2026-08-06

Project page: https://leloykun.github.io/ponder/steepest-descent-stiefel

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Terminology

• Stiefel Manifold — 14× in this paper · explained in 3 episodes
• Convergence Guarantees — 6× in this paper · explained in 3 episodes
• Muon Optimizer — 3× in this paper · explained in 2 episodes
• Objective Function — 3× in this paper · explained in 2 episodes
• Computational Efficiency — 2× in this paper · explained in 2 episodes
• Generalization — 2× in this paper · explained in 14 episodes
• Riemannian Optimization — 2× in this paper · explained in 3 episodes
• Accuracy — 1× in this paper · explained in 1 episode
• Adam — 1× in this paper · explained in 3 episodes
• Computational overhead — 1× in this paper · explained in 2 episodes
• Core — 1× in this paper · explained in 2 episodes
• Diffusion Model — 1× in this paper · explained in 10 episodes
• Diffusion Models — 1× in this paper · explained in 9 episodes
• Nuclear Norm — 1× in this paper · explained in 2 episodes
• Regularization — 1× in this paper · explained in 3 episodes
• Spectral Geometry — 1× in this paper · explained in 3 episodes

Sources

• Manifold constrained steepest descent for smooth and closed-set optimization
• Intrinsic Muon: Spectral Optimization on Riemannian Matrix Manifolds
• OrthoFuse: Training-free Riemannian Fusion of Orthogonal Style-Concept Adapters for Diffusion Models
• PoLAR: Polar-Decomposed Low-Rank Adapter Representation
• Tensorion: A Tensor-Aware Generalization of the Muon Optimizer
• LoRA meets Riemannion: Muon Optimizer for Parametrization-independent Low-Rank Adapters
• The Polar Express: Optimal Matrix Sign Methods and Their Application to the Muon Algorithm
• Insights on Muon from Simple Quadratics

Related papers

• Lions and Muons: Optimization via Stochastic Frank-Wolfe under Heavy-Tailed Noise
• Adam-HNAG: A Convergent Reformulation of Adam with Accelerated Rate
• Incremental Learning in Mirror Flows
• Beyond Aggregate Drift: Optimal Client-Specific Windows in Federated Learning
• Lower Bounds for Randomized Linear-Oracle Optimization over Strongly Convex Sets
• Second-Order Optimization with Asynchronous Curvature