Optimal Oracle Complexity for Finite-Sum Monotone Inclusions
math.OC, cs.LG
Submitted: 2026-10-04
Updated: 2026-10-04
Comments: 21 pages, 1 figure, 2 tables
License: http://creativecommons.org/licenses/by/4.0/
The gist: We present an oracle-optimal method for finite-sum monotone inclusions under mean-square Lipschitz continuity.
Abstract
We present an oracle-optimal method for finite-sum monotone inclusions under mean-square Lipschitz continuity. Our switching regularization method finds a point y and a certificate g in G(y) with (E|F(y)+g| 2) 1/2 epsilon using O(n+ sqrt n LR/epsilon) expected component evaluations and resolvent evaluations. It removes the additive n n cost of restarting a variance-reduced solver at every regularization stage by switching to a centered stochastic proximal iteration at regularization strength L/sqrt n. Carrying an operator estimate between the remaining stages limits their total cost to O(n). A matching Ω(n+ sqrt n LR/epsilon) lower bound holds for randomized linear-span component-oracle algorithms with adaptive stopping and expected query budgets. Thus, for 0< epsilon LR/2, our method attains the optimal worst-case expected component complexity in this oracle model, up to universal constants.
Sources
- How to Make the Gradient Mapping Small for Constrained Stochastic Min-Max Problems and Beyond
- Accelerated Algorithms for Constrained Nonconvex-Nonconcave Min-Max Optimization and Comonotone Inclusion
- Variance-Reduced Fast Krasnoselkii-Mann Methods for Finite-Sum Root-Finding Problems
- New Accelerated Past-Extragradient Methods with Variance Reduction for Generalized Equations
- Unbiased and Biased Variance-Reduced Forward-Reflected-Backward Splitting Methods for Stochastic Composite Inclusions
- Direct Acceleration of Stochastic Root-Finding Without Variance Reduction and Regularization
Related papers
- Lions and Muons: Optimization via Stochastic Frank-Wolfe under Heavy-Tailed Noise
- Adam-HNAG: A Convergent Reformulation of Adam with Accelerated Rate
- Incremental Learning in Mirror Flows
- Second-Order Optimization with Asynchronous Curvature
- Online Control via Counterfactual Tracking
- Asynchronous Replanning in Two Population Linear Quadratic Mean Field Games: Information Requirements and Stability