Kinetic theory for Transformers and the lost-in-the-middle phenomenon

arXiv:2605.09213 · math.AP, cs.LG, math.PR · Submitted 2026-05-09 · Read on arXiv

math.AP, cs.LG, math.PR

Submitted: 2026-05-09

Updated: 2026-10-20

Terminology

• Transformer — 2× in this paper · explained in 2 episodes
• Transformers — 2× in this paper · explained in 2 episodes
• Decoder — 1× in this paper · explained in 2 episodes

Related papers

• Anisotropic uncertainty principles for metaplectic operators
• Convergence of graph Dirichlet energies and graph Laplacians on intersecting manifolds of varying dimensions
• Fractional Parabolic Partial Differential Equations in Anisotropic Spectral Barron Spaces: Regularity and Neural Approximation
• Regularity of Second-Order Elliptic PDEs in Spectral Barron Spaces
• Determining a parabolic-elliptic-elliptic system by boundary observation of its non-negative solutions under chemotaxis background
• On the basic reproduction number in continuously structured populations