First-Token Attraction in Mamba Dynamics
cs.AI
Submitted: 2026-11-05
Updated: 2026-11-05
Terminology
Sources
- Clustering in pure-attention hardmax transformers and its role in sentiment analysis
- The Hidden Attention of Mamba Models
- Layer Normalization
- Self-attention Networks Localize When QK-eigenspectrum Concentrates
- Interpolation, Approximation and Controllability of Deep Neural Networks
- Setting the Record Straight on Transformer Oversmoothing
- Dynamic metastability in the self-attention model
- A mathematical perspective on Transformers
- ContraNorm: A Contrastive Learning Perspective on Oversmoothing and Beyond
- Time-Parameterized Convolutional Neural Networks for Irregularly Sampled Time Series
- From Generalization Analysis to Optimization Designs for State Space Models
- Understanding and Improving Transformer From a Multi-Particle Dynamic System Point of View
- Decision Mamba: Reinforcement Learning via Sequence Modeling with Selective State Spaces
- Mamba-360: Survey of State Space Models as Transformer Alternative for Long Sequence Modelling: Methods, Applications, and Challenges
- MoE-Mamba: Efficient Selective State Space Models with Mixture of Experts
- Residual Connections and Normalization Can Provably Prevent Oversmoothing in GNNs
- Graph-Mamba: Towards Long-Range Graph Sequence Modeling with Selective State Spaces
- Mamba-UNet: UNet-Like Pure Visual Mamba for Medical Image Segmentation
- Visual Mamba: A Survey and New Outlooks
- Vivim: a Video Vision Mamba for Medical Video Segmentation
Related papers
- Multi-Objective Structured Pruning of LLMs for Latency and Model Size Optimization
- MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
- Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
- Embeddings for Preferences, Not Semantics
- Can We Trust Item Response Theory for AI Evaluation?
- Diagnosing Task Insensitivity in Language Agents