Perturbed Iterate SGD for Lipschitz Continuous Loss Functions with Numerical Error and Adaptive Step Sizes

Name: Perturbed Iterate SGD for Lipschitz Continuous Loss Functions with Numerical Error and Adaptive Step Sizes
Uploaded: 2026-10-17T23:48:50.743443+00:00
Duration: 349 s
Description: This work analyzes a modified SGD algorithm (PISGD) that uses adaptive step sizes and handles numerical errors in stochastic gradient computation for Lipschitz continuous loss functions. It proves asymptotic convergence to a Clarke stationary point almost surely, and provides non-asymptotic convergence guarantees in expectation, showing the algorithm reliably finds approximate stationary points.

This work analyzes a modified SGD algorithm (PISGD) that uses adaptive step sizes and handles numerical errors in stochastic gradient computation for Lipschitz continuous loss functions. It proves asymptotic convergence to a Clarke stationary point almost surely, and provides non-asymptotic convergence guarantees in expectation, showing the algorithm reliably finds approximate stationary points.

Open the full episode