Diagonal Equivalence in State Space Models: Characterization, Retrieval Rank, and Selective Extensions
cs.LG, math.OC, stat.ML
Submitted: 2026-11-02
Updated: 2026-11-02
Terminology
Sources
- The Hidden Attention of Mamba Models
- Zoology: Measuring and Improving Recall in Efficient Language Models
- Language Models Enable Simple Systems for Generating Structured Views of Heterogeneous Data Lakes
- Just read twice: closing the recall gap for recurrent language models
- Tiled Flash Linear Attention: More Efficient Linear RNN and xLSTM Kernels
- PIQA: Reasoning about Physical Commonsense in Natural Language
- Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge
- Griffin: Mixing Gated Linear Recurrences with Local Attention for Efficient Language Models
- How Many Heads Make an SSM? A Unified Framework for Attention and State Space Models
- RULER: What's the Real Context Size of Your Long-Context Language Models?
- On Structured State-Space Duality
- GateLoop: Fully Data-Controlled Linear Recurrence for Sequence Modeling
- Long Range Language Modeling via Gated State Spaces
- Selective Rotary Position Embedding
- RWKV-7 "Goose" with Expressive Dynamic State Evolution
- HGRN2: Gated Linear RNNs with State Expansion
- WinoGrande: An Adversarial Winograd Schema Challenge at Scale
- DeltaProduct: Improving State-Tracking in Linear RNNs via Householder Products
- RoFormer: Enhanced Transformer with Rotary Position Embedding
- Retentive Network: A Successor to Transformer for Large Language Models
Related papers
- A Physics-Regulated Neural Framework for Learning 3D Grain Growth Dynamics
- FlexMoE: Learning One-for-All Multi-Budget Expert Pruning for Flexible MoE Deployment
- OptiMind: Improving Small Language Models for Optimization Modeling through Formulation Guidance and Multi-Turn Learning
- Staying in the Sweet Spot: Boost Reasoning Evolution via Responsive Rollout Difficulty Manipulation
- Examining the Impact of Data and Training Methods on Emergent and Subliminal Misalignment
- Rethinking Black-Box On-Policy Distillation: A Rubric-based Analysis