SPARE: Fast Linear Attention Decoding with Compact Recurrent States
cs.CL
Submitted: 2026-11-05
Updated: 2026-11-05
Terminology
Sources
- Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Duality
- A Persistent-State Dataflow Accelerator for Memory-Bound Linear Attention Decode on FPGA
- Gated DeltaNet-2: Decoupling Erase and Write in Linear Attention
- On State Reduction in Linear Attention
- LeapQuant: Efficient Linear Attention with Accurate Recurrent State Quantization
- Lightning Attention-2: A Free Lunch for Handling Unlimited Sequence Lengths in Large Language Models
- State Rank Dynamics in Linear Attention LLMs
- Kimi Linear: An Expressive, Efficient Attention Architecture
- Gated Linear Attention Transformers with Hardware-Efficient Training
Related papers
- ALARM: Audio-Language Alignment for Reasoning Models
- ThinkSum: Probabilistic reasoning over sets using large language models
- Coherence boosting: When your pretrained language model is not paying enough attention
- From Facts to Insights: A Persona-Driven Dual Memory Framework and Dataset for Role-Playing Agents
- Improving General Role-Playing Agents via Psychology-Grounded Reasoning and Role-Aware Policy Optimization
- The Art of Asking: Multilingual Prompt Optimization for Synthetic Data