Intrinsic Associative Memory on Riemannian Manifolds: Curvature, Capacity, and Emergent Modes
cs.LG, cs.AI, stat.ML
Submitted: 2026-09-28
Updated: 2026-09-28
License: http://creativecommons.org/licenses/by/4.0/
The gist: Geometry does more than constrain an associative memory: curvature determines what it remembers and which states it creates.
Terminology
Abstract
Geometry does more than constrain an associative memory: curvature determines what it remembers and which states it creates. We develop intrinsic dense associative memories on Riemannian manifolds by casting memory as Epanechnikov kernel-density mode seeking. We compare geodesic and volume-corrected energies and show that curvature separates their behavior. We prove that geodesic memory always retains an isolated pattern, while corrected memory obeys a sharp Ricci-curvature threshold: positive curvature can erase memories in high dimensions, while negative curvature reinforces them. We derive geodesic capacity scalings of q β-1/2 for retaining every pattern and q β-1 for a typical one, where q β is the pairwise kernel-overlap probability. We show how overlap creates novel memories: designed N-pattern configurations realize all 2 N-1 subset modes, but random data at the storage threshold yield only a Poisson number. We establish exact one-step recall using Riemannian mean shift. In simulations, we recover the predicted curvature transition and every designed mode. On WordNet's full noun hierarchy, we demonstrate that volume correction improves low-capacity retrieval. Together, our work shows that curvature is a design variable for associative memory, not merely a property of the data.
Sources
- On The Convergence of Gradient Descent for Finding the Riemannian Center of Mass
- Density Estimation on Manifolds with Boundary
- On a model of associative memory with huge storage capacity
- Position: Beyond Euclidean -- Foundation Models Should Embrace Non-Euclidean Geometries
- Dense Associative Memory with Epanechnikov Energy
- On Sparse Modern Hopfield Model
- Dense Associative Memory for Pattern Recognition
- Text Has Curvature
Related papers
- Polynomial-Augmented Neural Networks (PANNs) with Weak Orthogonality Constraints for Enhanced Function and PDE Approximation
- AIRL-S: Unifying Reinforcement Learning and Search-Based Test-Time Scaling via Adversarial Inverse Reinforcement Learning
- Transformers as Bayesian In-Context Experimenters: Smoothness-Adaptive Efficient ATE Estimation
- Convergence issues in Relational Concept Analysis based on AOC-posets
- Beliefs Beyond Posteriors: The Predictive Cost of ELBO Binding
- Understanding Diffusion Models via Ratio-Based Function Approximation with SignReLU Networks