Finite Spectral Depth Controls Gaussian Entrance Rank and Avoidance of the First Gradient-Flow Saddle
cs.LG, math.OC
Submitted: 2026-11-03
Updated: 2026-11-03
Terminology
Sources
- Old Optimizer, New Norm: An Anthology
- Saddle-to-Saddle Dynamics in Deep Linear Networks: Small Initialization Training, Symmetry, and Sparsity
- Kimi K2: Open Agentic Intelligence
- Early Directional Convergence in Deep Homogeneous Neural Networks for Small Initializations
- Muon with Finite Newton-Schulz: The Smoothing Benefit in Nonsmooth Nonconvex Optimization
- Last-Iterate Stationarity of Muon with Finitely Many Newton-Schulz Iterations
- Muon Learns Facts Better: Understanding the Role of Spectral Orthogonalization
- Muon is Scalable for LLM Training
- Preconditioning Benefits of Spectral Orthogonalization in Muon
- The Zero Set of a Real Analytic Function
- Convergence of Practical Muon with Finite Newton-Schulz Iterations and Nesterov Momentum
- Muon learns balanced solutions in matrix factorization without slow saddle-to-saddle dynamics
- Exact solutions to the nonlinear dynamics of learning in deep linear neural networks
Related papers
- A Physics-Regulated Neural Framework for Learning 3D Grain Growth Dynamics
- FlexMoE: Learning One-for-All Multi-Budget Expert Pruning for Flexible MoE Deployment
- OptiMind: Improving Small Language Models for Optimization Modeling through Formulation Guidance and Multi-Turn Learning
- Staying in the Sweet Spot: Boost Reasoning Evolution via Responsive Rollout Difficulty Manipulation
- Examining the Impact of Data and Training Methods on Emergent and Subliminal Misalignment
- Rethinking Black-Box On-Policy Distillation: A Rubric-based Analysis