Stochastic gradient descent with discontinuity across a manifold

arXiv:2608.07618 · stat.ML, cs.LG, math.PR · Submitted 2026-08-07 · Read on arXiv

Vivek S. Borkar

stat.ML, cs.LG, math.PR

Submitted: 2026-08-07

Updated: 2026-08-11

Comments: 13 pages

License: http://creativecommons.org/licenses/by/4.0/

The gist: Stochastic gradient descent for a loss function discontinuous across lower dimensional manifolds is analyzed by studying its differential equation limit.

Terminology

• Stochastic Gradient Descent — 8× in this paper
• Loss Function — 4× in this paper · explained in 2 episodes
• Gradient Flow — 2× in this paper · explained in 2 episodes
• Stochastic Gradient Descent (SGD) — 1× in this paper · explained in 5 episodes

Abstract

Sources

• The global convergence time of stochastic gradient descent in non-convex landscapes: Sharp estimates via large deviations

Related papers

• Behavior of prediction performance metrics with rare events
• Optimal Estimation of Generic Dynamics by Path-Dependent Neural Jump ODEs
• A Posterior-Dynamics Framework for Imaging Inverse Problems with Pretrained Diffusion Priors
• Scalable Posterior Uncertainty for Flexible Density-Based Clustering
• A Transport Geometry of Benign Overfitting
• Efficient quadratic entropy with distance sketches