Gradient Descent's Last Iterate is Often (slightly) Suboptimal
math.OC, cs.LG
Submitted: 2026-04-15
Updated: 2026-10-02
Sources
- Fast Last-Iterate Convergence of SGD in the Smooth Interpolation Regime
- Last-Iterate Complexity of SGD for Convex and Smooth Stochastic Problems
- A simpler approach to obtaining an O(1/t) convergence rate for the projected stochastic subgradient method
- On the Last-Iterate Convergence of Shuffling Gradient Methods
- Exact convergence rate of the last iterate in subgradient methods
Related papers
- Lions and Muons: Optimization via Stochastic Frank-Wolfe under Heavy-Tailed Noise
- Adam-HNAG: A Convergent Reformulation of Adam with Accelerated Rate
- Incremental Learning in Mirror Flows
- Second-Order Optimization with Asynchronous Curvature
- Online Control via Counterfactual Tracking
- Asynchronous Replanning in Two Population Linear Quadratic Mean Field Games: Information Requirements and Stability