Entropy-Smooth Convex Optimization Cannot Be Accelerated
Jacob M. Aguirre, Dmitrii M. Ostrovskii
math.OC, cs.IT, stat.ML
Submitted: 2026-07-29
Comments: 19 pages
License: http://creativecommons.org/licenses/by/4.0/
The gist: We prove an (L/T) lower bound for the convergence rate of minimization in the class of functions that are convex and L-smooth relative to negative entropy on the standard d-simplex, valid for every
Terminology
Abstract
We prove an (L/T) lower bound for the convergence rate of minimization in the class of functions that are convex and L-smooth relative to negative entropy on the standard d-simplex, valid for every first-order method when d = (T 2). In particular, this shows that mirror descent is optimal up to a logarithmic factor in this class. This may be surprising due to the fact that accelerated methods are readily available under the assumption of smoothness in 1-norm. While Dragomir et al. (Mathematical Programming, 2022) have already showed that acceleration might be impossible under relative smoothness, their prox-function is pathological and constructed together with the hard instance. In contrast, we show non-acceleration for a specific prox-function with particularly favorable structure. We also extend the result to the quantum setting, proving the same lower bound in the class of functions L-smooth relative to negative von Neumann entropy on the spectrahedron of d times d Hermitian positive-semidefinite matrices with unit trace.
Related papers
- Lions and Muons: Optimization via Stochastic Frank-Wolfe under Heavy-Tailed Noise
- Adam-HNAG: A Convergent Reformulation of Adam with Accelerated Rate
- Incremental Learning in Mirror Flows
- Beyond Aggregate Drift: Optimal Client-Specific Windows in Federated Learning
- Lower Bounds for Randomized Linear-Oracle Optimization over Strongly Convex Sets
- Second-Order Optimization with Asynchronous Curvature