Mean--Fluctuation Dynamics at the Edge of Stability

arXiv:2606.05326 · math.OC, cs.AI, cs.LG, math-ph, math.AP, math.MP · Submitted 2026-06-03 · Read on arXiv

math.OC, cs.AI, cs.LG, math-ph, math.AP, math.MP

Submitted: 2026-06-03

Updated: 2026-09-30

Terminology

• Accuracy — 1× in this paper · explained in 1 episode
• Gradient Flow — 1× in this paper · explained in 2 episodes

Sources

• The Multiple Timescales of Gradient Descent on the Edge of Stability: A Perturbative Derivation of the Central Flow
• Quantitative Convergence of Wasserstein Gradient Flows of Kernel Mean Discrepancies
• Wasserstein gradient flows for Coulomb discrepancies
• Gradient Descent on Neural Networks Typically Occurs at the Edge of Stability
• Understanding Optimization in Deep Learning with Central Flows
• The large learning rate phase of deep learning: the catapult mechanism
• Edge Flow: A Tractable and Predictive Continuous-Time Model for Gradient Descent at the Edge of Stability
• A Rod Flow Model for Adam at the Edge of Stability
• Rod Flow: A Continuous-Time Model for Gradient Descent at the Edge of Stability
• Wasserstein gradient flows of Maximum Mean Discrepancy with energy kernels
• Understanding Edge-of-Stability Training Dynamics with a Minimalist Example

Related papers

• Lions and Muons: Optimization via Stochastic Frank-Wolfe under Heavy-Tailed Noise
• Adam-HNAG: A Convergent Reformulation of Adam with Accelerated Rate
• Incremental Learning in Mirror Flows
• Second-Order Optimization with Asynchronous Curvature
• Online Control via Counterfactual Tracking
• Asynchronous Replanning in Two Population Linear Quadratic Mean Field Games: Information Requirements and Stability