Nonreciprocal Spherical Attention and Retrieval: Sharp Coarsening Clocks and Certified Score Correction
cs.LG, cs.IR, math.DS
Submitted: 2026-11-01
Updated: 2026-11-01
Comments: 42 pages, 9 figures, 10 tables. Includes mathematical appendices and ancillary reproducibility code
Code: https://github.com/erikbern/ann-benchmarks
License: http://creativecommons.org/licenses/by/4.0/
The gist: We develop nonreciprocal spherical attention as a mathematical framework for coarsening and compressed nearest-neighbor retrieval.
Terminology
Abstract
We develop nonreciprocal spherical attention as a mathematical framework for coarsening and compressed nearest-neighbor retrieval. For two shrinking acute blocks whose centers have zero skew pairing and whose total mass ratio is subexponential in inverse temperature, we prove β to infinityβ-1 T β=1- ϕ, without restricting particle-count growth. An explicit three-body flow has two rigorously separated metastable exponents. For finite compression, mass centering yields a sharp O(r 2) attention-output error bound at fixed temperature, with an explicit spherical-normalization correction. We formulate skew query-side score calibration as a strictly convex moment problem and derive a unique normal equation, residual-based optimization certificates, and bounded low-rank projections. A norm-aware margin objective calibrates native squared-distance rankings. Experiments use semantic retrieval data, one million visual descriptors with three codebook seeds, and a ten-million-vector FAISS IVF-PQ index with recomputed exact cosine ground truth. On SIFT1M, skew margin calibration raises native-L2 hit@10 from 0.8926 to 0.8948; symmetric calibration reaches 0.8996. The ten-million-vector index reaches 0.8827 hit@10 at 0.1794 ms per query in a four-thread batch benchmark, while validation selects no skew update. Exact streamed attention over ten million keys evaluates compressed memory on 1,024 held-out queries. Independent matrix solves, automatic differentiation, finite differences, interval arithmetic, full-flow tests, and nine Rocq-checked algebra lemmas support reproducibility. Directionality alone does not establish retrieval superiority.
Sources
- The emergence of clusters in self-attention dynamics
- A mathematical perspective on Transformers
- Dynamic metastability in the self-attention model
- The Mean-Field Dynamics of Transformers
- Spectral Selection in Symmetric Self-Attention Dynamics
- The Routing and Filtering Structure of Attention
- Orthogonal Self-Attention
- Attention is Not All You Need: Pure Attention Loses Rank Doubly Exponentially with Depth
- The EM Perspective of Directional Mean Shift Algorithm
- Time-inhomogeneous diffusion geometry and topology
- Almost global consensus on the n-sphere
- A Lifting method for analyzing distributed synchronization on the unit sphere
- Token Merging: Your ViT But Faster
- Asymmetric LSH (ALSH) for Sublinear Time Maximum Inner Product Search (MIPS)
- LoRANN: Low-Rank Matrix Factorization for Approximate Nearest Neighbor Search
- BEIR: A Heterogenous Benchmark for Zero-shot Evaluation of Information Retrieval Models
- MTEB: Massive Text Embedding Benchmark
- Optimization methods for solving matrix equations
- The Faiss library
- Jointly Optimizing Query Encoder and Product Quantization to Improve Retrieval Performance
Related papers
- A Physics-Regulated Neural Framework for Learning 3D Grain Growth Dynamics
- FlexMoE: Learning One-for-All Multi-Budget Expert Pruning for Flexible MoE Deployment
- OptiMind: Improving Small Language Models for Optimization Modeling through Formulation Guidance and Multi-Turn Learning
- Staying in the Sweet Spot: Boost Reasoning Evolution via Responsive Rollout Difficulty Manipulation
- Examining the Impact of Data and Training Methods on Emergent and Subliminal Misalignment
- Rethinking Black-Box On-Policy Distillation: A Rubric-based Analysis