Accelerated Gradient Methods Through Variable and Operator Splitting
math.OC
Submitted: 2025-05-07
Updated: 2026-10-30
License: http://creativecommons.org/licenses/by/4.0/
The gist: This survey develops a variable and operator splitting (VOS) framework for accelerated first-order methods.
Terminology
Abstract
This survey develops a variable and operator splitting (VOS) framework for accelerated first-order methods. Operator splitting separates the components of the optimality system, while variable splitting introduces auxiliary dynamics that generate acceleration. The strong Lyapunov property guides the stability analysis, and practical schemes follow from Accelerated Over-Relaxation (AOR) and Extrapolation by Predictor--Corrector (EPC) discretizations. For merely convex problems, a perturbed VOS flow combined with homotopy in the auxiliary scale yields accelerated rates. The framework gives a common derivation and analysis of Nesterov-type, FISTA-type, composite, mirror-descent-type, and bilinear saddle-point accelerated methods. Its main contribution is structural and expository: it organizes these methods within one Lyapunov and splitting template and explains how acceleration survives different splittings and discretizations.
Sources
- A Unified Convergence Analysis of First Order Convex Optimization Methods via Strong Lyapunov Functions
- Accelerated Gradient and Skew-Symmetric Splitting Methods for a Class of Monotone Operator Equations
- Provable non-accelerations of the heavy-ball method
Related papers
- An Accelerated Proximal Gradient Method for Bures-Wasserstein Space
- FOMTools: Development and AI-Assisted Code Generation for First-Order Methods
- Lions and Muons: Optimization via Stochastic Frank-Wolfe under Heavy-Tailed Noise
- Dynamic Service Recommendation with Congestion-Dependent Joining: Near-Optimal and Constant-Factor Approximation Algorithms
- Integrating Learning and Optimization for Open Transport Capacity Pooling under Disruption
- Zeroth-Order Gradient Estimators under Decision-Dependent Noise