Everything in its right place: attention is a radio head
physics.soc-ph, cond-mat.dis-nn, cs.AI, cs.LG, nlin.AO
Submitted: 2026-11-09
Updated: 2026-11-09
Code: https://github.com/mystic-blue/radiohead
Terminology
Sources
- Scaling Laws for Neural Language Models
- Training Compute-Optimal Large Language Models
- Intuition emerges in Maximum Caliber models at criticality
- Kuramoto Attention: Synchronizing Self-Attention on the Torus
- Attention as Frustrated Synchronization
- Krause Synchronization Transformers
- Selective Synchronization Attention
- TinyStories: How Small Can Language Models Be and Still Speak Coherent English?
Related papers
- Transient Signal Propagation in Heterogeneous Networks under Various Inputs
- Phase Transitions in Economic Inequality:Taxation and Extremal Replacement Dynamics
- New rules, old habits: across seven team sports, each league keeps its own way of using the rules
- URBAN-SPIN: A street-level bikeability index to inform design implementations in historical city centres
- Logarithmic dispersion at marginal stability in collective attention
- Glassy dynamics of metropolitan human mobility frozen near the gravitational equilibrium