Synchronization on circles and spheres with nonlinear interactions

arXiv:2405.18273 · math.OC, cs.LG, math.DS · Submitted 2026-01-27 · Read on arXiv

Christopher Criscitiello, Quentin Rebjock, Andrew D. McRae, Nicolas Boumal

math.OC, cs.LG, math.DS

Submitted: 2026-01-27

Updated: 2026-08-24

Comments: 30 pages, 1 figure

DOI: 10.1137/24M1699759

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Terminology

• Gradient Flow — 20× in this paper · explained in 2 episodes
• Transformer — 16× in this paper · explained in 2 episodes
• Conference on Decision and Control — 3× in this paper
• Adjacency Matrix — 2× in this paper
• Clustering — 2× in this paper · explained in 2 episodes
• Representation Learning — 2× in this paper · explained in 3 episodes
• Ambient Dimension — 1× in this paper
• Entrainment — 1× in this paper · explained in 2 episodes
• Inverse Problem — 1× in this paper · explained in 2 episodes
• Metastability — 1× in this paper · explained in 3 episodes
• Riemannian optimization — 1× in this paper · explained in 3 episodes
• Sparsity — 1× in this paper · explained in 5 episodes
• Stiefel manifold — 1× in this paper · explained in 2 episodes
• Transformer Architecture — 1× in this paper · explained in 3 episodes
• Transformer Models — 1× in this paper · explained in 2 episodes

Sources

• Dynamic metastability in the self-attention model
• Saddle-to-Saddle Dynamics in Deep Linear Networks: Small Initialization Training, Symmetry, and Sparsity
• Understanding and Improving Transformer From a Multi-Particle Dynamic System Point of View
• Synchronization of mean-field models on the circle
• The Asymptotic Behavior of Attention in Transformers
• Synchronization on the circle

Related papers

• Lions and Muons: Optimization via Stochastic Frank-Wolfe under Heavy-Tailed Noise
• Adam-HNAG: A Convergent Reformulation of Adam with Accelerated Rate
• Incremental Learning in Mirror Flows
• Beyond Aggregate Drift: Optimal Client-Specific Windows in Federated Learning
• Lower Bounds for Randomized Linear-Oracle Optimization over Strongly Convex Sets
• Second-Order Optimization with Asynchronous Curvature