Stochastic learning flow: moment equations for stochastic gradient descent at finite step size
cs.LG, cond-mat.dis-nn
Submitted: 2026-10-30
Updated: 2026-10-30
Terminology
Sources
- Exact solutions to the nonlinear dynamics of learning in deep linear neural networks
- Three Factors Influencing Minima in SGD
- The large learning rate phase of deep learning: the catapult mechanism
- Implicit Gradient Regularization
- Edge of Stochastic Stability: Revisiting the Edge of Stability for SGD
- SGD at the Edge of Stability: The Stochastic Sharpness Gap
- Gradient Descent Happens in a Tiny Subspace
- Saddle-to-Saddle Dynamics in Deep Linear Networks: Small Initialization Training, Symmetry, and Sparsity
- Type-II Saddles and Probabilistic Stability of Stochastic Gradient Descent
- Stochastic Modified Flows, Mean-Field Limits and Dynamics of Stochastic Gradient Descent
Related papers
- A Physics-Regulated Neural Framework for Learning 3D Grain Growth Dynamics
- FlexMoE: Learning One-for-All Multi-Budget Expert Pruning for Flexible MoE Deployment
- OptiMind: Improving Small Language Models for Optimization Modeling through Formulation Guidance and Multi-Turn Learning
- Staying in the Sweet Spot: Boost Reasoning Evolution via Responsive Rollout Difficulty Manipulation
- Examining the Impact of Data and Training Methods on Emergent and Subliminal Misalignment
- Rethinking Black-Box On-Policy Distillation: A Rubric-based Analysis