Establishing Boundary KKT Convergence of Mirror Descent through Reparameterization
math.OC, cs.LG
Submitted: 2026-08-07
Updated: 2026-08-28
Comments: 23 pages; v2 adds a discussion of concurrent independent work and clarifies comparisons with related preprints
License: http://creativecommons.org/licenses/by-nc-nd/4.0/
The gist: Sequence convergence to a boundary Karush--Kuhn--Tucker (KKT) point has long remained unclear for nonconvex mirror descent with Legendre kernels.
Terminology
Abstract
Sequence convergence to a boundary Karush--Kuhn--Tucker (KKT) point has long remained unclear for nonconvex mirror descent with Legendre kernels. The difficulty arises from the blow-up of the gradient of the Legendre kernel at the boundary. Recent work shows that mirror descent can accumulate at non-KKT boundary points despite decreasing objective values, precluding a convergence guarantee to KKT points in general. Despite this negative result, mirror descent remains effective in many real applications. Motivated by this contrast, we address the boundary difficulty directly and establish KKT convergence of mirror descent for a broad class of structured nonconvex problems. We analyze mirror descent in reparameterized variables, where the Hessian metric is flattened and remains nondegenerate as the boundary is approached. Under extension and definability conditions jointly coupling the objective, the Legendre kernel, and the feasible region, the reparameterized sequence has finite length and converges, thereby recovering convergence to a KKT point of the original sequence. Our general framework applies to some concrete instances: Shannon entropy, Fermi--Dirac entropy, and power kernels on polyhedron.
Sources
- A Unified Framework for Iterate Convergence of Bregman Proximal Methods
- Spurious Stationarity and Hardness Results for Bregman Proximal-Type Algorithms
- On the Iterate Convergence of Bregman Projected Gradient Method
- On exploration of an interior mirror descent flow for stochastic nonconvex constrained problem
- Non-KKT Accumulation in Entropic Mirror Descent
- A Bregman ADMM for Bethe variational problem
- Shuffling the Stochastic Mirror Descent via Dual Lipschitz Continuity and Kernel Conditioning
- On the Convergence Rate of Stochastic Mirror Descent for Nonsmooth Nonconvex Optimization
Related papers
- Lions and Muons: Optimization via Stochastic Frank-Wolfe under Heavy-Tailed Noise
- Adam-HNAG: A Convergent Reformulation of Adam with Accelerated Rate
- Incremental Learning in Mirror Flows
- Beyond Aggregate Drift: Optimal Client-Specific Windows in Federated Learning
- Lower Bounds for Randomized Linear-Oracle Optimization over Strongly Convex Sets
- Second-Order Optimization with Asynchronous Curvature