The Row Normalization Puzzle in Muon
cs.LG, math.OC
Submitted: 2026-09-30
Updated: 2026-09-30
Sources
- Dion: Distributed Orthonormalized Updates
- Dion3: Full-Stack Orthogonal Updates
- MuonEq: Balancing Before Orthogonalization with Lightweight Equilibration
- Muon with Nesterov Momentum: Heavy-Tailed Noise and (Randomized) Inexact Polar Decomposition
- Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge
- When do spectral gradient updates help in deep learning?
- Aurora: A Leverage-Aware Spectral Optimizer
- Insights on Muon from Simple Quadratics
- Muown Implicitly Performs Angular Step-size Decay
- 94% on CIFAR-10 in 3.29 Seconds on a Single GPU
- Toto 2.0: Time Series Forecasting Enters the Scaling Era
- One weird trick for parallelizing convolutional neural networks
- PolarGrad: A Class of Matrix-Gradient Optimizers from a Unifying Preconditioning Perspective
- Muown: Row-Norm Control for Muon Optimization
- Muon is Scalable for LLM Training
- Muon$^2$: Boosting Muon via Adaptive Second-Moment Preconditioning
- Preconditioning Benefits of Spectral Orthogonalization in Muon
- Muon Does Not Converge on Convex Lipschitz Functions
- AdaMuon: Adaptive Muon Optimizer
Related papers
- Polynomial-Augmented Neural Networks (PANNs) with Weak Orthogonality Constraints for Enhanced Function and PDE Approximation
- AIRL-S: Unifying Reinforcement Learning and Search-Based Test-Time Scaling via Adversarial Inverse Reinforcement Learning
- Transformers as Bayesian In-Context Experimenters: Smoothness-Adaptive Efficient ATE Estimation
- Convergence issues in Relational Concept Analysis based on AOC-posets
- Beliefs Beyond Posteriors: The Predictive Cost of ELBO Binding
- Understanding Diffusion Models via Ratio-Based Function Approximation with SignReLU Networks