AMUSE: Anytime Muon with Stable Gradient Evaluation
cs.LG
Submitted: 2026-05-21
Updated: 2026-11-13
Code: https://github.com/kjeiun/amuse
Terminology
Sources
- Anytime Training with Schedule-Free Spectral Optimization
- Skin Lesion Analysis Toward Melanoma Detection 2018: A Challenge Hosted by the International Skin Imaging Collaboration (ISIC)
- Benchmarking Neural Network Training Algorithms
- Self-Stabilization: The Implicit Bias of Gradient Descent at the Edge of Stability
- ScheduleFree+: Scaling Learning-Rate-Free & Schedule-Free Learning to Large Language Models
- GLM-5: from Vibe Coding to Agentic Engineering
- ROOT: Robust Orthogonalized Optimizer for Neural Network Training
- Uniform Spectral Growth and Convergence of Muon in LoRA-Style Matrix Factorization
- Kimi K2: Open Agentic Intelligence
- NorMuon: Making Muon more efficient and scalable
- Muon is Scalable for LLM Training
- FOCUS: First Order Concentrated Updating Scheme
- REG: A Regularization Optimizer for Robust Training Dynamics
- Preconditioning Benefits of Spectral Orthogonalization in Muon
- Connections between Schedule-Free Optimizers, AdEMAMix, and Accelerated SGD Variants
- The Full Spectrum of Deepnet Hessians at Scale: Dynamics with SGD Training and Sample Size
- Delving into Muon and Beyond: Deep Analysis and Extensions
- Can Muon Fine-tune Adam-Pretrained Models?
- Eigenvalues of the Hessian in Deep Learning: Singularity and Beyond
- Empirical Analysis of the Hessian of Over-Parametrized Neural Networks
Related papers
- A Physics-Regulated Neural Framework for Learning 3D Grain Growth Dynamics
- FlexMoE: Learning One-for-All Multi-Budget Expert Pruning for Flexible MoE Deployment
- OptiMind: Improving Small Language Models for Optimization Modeling through Formulation Guidance and Multi-Turn Learning
- Staying in the Sweet Spot: Boost Reasoning Evolution via Responsive Rollout Difficulty Manipulation
- Examining the Impact of Data and Training Methods on Emergent and Subliminal Misalignment
- Rethinking Black-Box On-Policy Distillation: A Rubric-based Analysis