DGA-Muon: Decoupled Geometry-Aligned Adaptive Scaling for Muon
cs.AI
Submitted: 2026-10-05
Updated: 2026-10-07
Code: https://github.com/KellerJordan/modded-nanogpt
Sources
- Accelerating Newton-Schulz Iteration for Orthogonalization via Chebyshev-type Polynomials
- ROOT: Robust Orthogonalized Optimizer for Neural Network Training
- A Note on the Convergence of Muon
- Muon is Scalable for LLM Training
- Muon$^2$: Boosting Muon via Adaptive Second-Moment Preconditioning
- Spectral Flattening Is All Muon Needs: How Orthogonalization Controls Learning Rate and Convergence
- Practical Efficiency of Muon for Pretraining
- Decoupling Variance and Scale-Invariant Updates in Adaptive Gradient Descent for Unified Vector and Matrix Optimization
- Isotropic Curvature Model for Understanding Deep Learning Optimization: Is Gradient Orthogonalization Optimal?
- Kimi K2: Open Agentic Intelligence
- Kimi K3: Open Frontier Intelligence
- Why Muon Outperforms Adam: A Curvature Perspective
- DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligence
- FISMO: Fisher-Structured Momentum-Orthogonalized Optimizer
- GLM-4.5: Agentic, Reasoning, and Coding (ARC) Foundation Models
- GLM-5: from Vibe Coding to Agentic Engineering
- Matrix AdaGrad: Row-wise and Column-wise Adaptive Subgradient Methods
- Mousse: Rectifying the Geometry of Muon with Curvature-Aware Preconditioning
Related papers
- MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
- Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
- Agents' Last Exam
- The Clinician's Veto: Navigating Trust, Liability, and Uncertainty in Autonomous AI Prescribing
- MindHelper: Closed-Loop Embodied Mental-State Reasoning for Precision Intervention
- Incumbent Advantage: Brand Bias and Cognitive Manipulation Dynamics in LLM Recommendation Systems