SinkState: On-the-Fly SVD of Recurrent States for Efficient Hybrid LLM Serving
cs.LG
Submitted: 2026-11-10
Updated: 2026-11-10
Code: https://github.com/fla-org/flash-linear-attention
Terminology
Sources
- Palu: Compressing KV-Cache with Low-Rank Projection
- xKV: Cross-Layer KV-Cache Compression via Aligned Singular Vector Extraction
- Quamba2: A Robust and Scalable Post-training Quantization Framework for Selective State Space Models
- Quamba: A Post-Training Quantization Recipe for Selective State Space Models
- FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness
- Mamba: Linear-Time Sequence Modeling with Selective State Spaces
- Measuring Mathematical Problem Solving With the MATH Dataset
- RULER: What's the Real Context Size of Your Long-Context Language Models?
- Transformers are RNNs: Fast Autoregressive Transformers with Linear Attention
- Kimi Linear: An Expressive, Efficient Attention Architecture
- Kimi K3: Open Frontier Intelligence
- SketchSSM: Write to the Full State, Read from a Compact Sketch
- Efficient Memory Management for Large Language Model Serving with PagedAttention
- Jamba: A Hybrid Transformer-Mamba Language Model
- Let's Verify Step by Step
- AWQ: Activation-aware Weight Quantization for LLM Compression and Acceleration
- ReplaySSM: Cache SSM Inputs, Not State
- LinearKV: One Cached State Suffices for Position-Independent Caching in Hybrid LLMs
- Just Let Linear States Forget the Distant Past: Prefix Caching via Suffix Replay for Hybrid LLMs
- HYPIC: Position-Independent Caching for Hybrid-Attention LLM Serving
Related papers
- A Physics-Regulated Neural Framework for Learning 3D Grain Growth Dynamics
- FlexMoE: Learning One-for-All Multi-Budget Expert Pruning for Flexible MoE Deployment
- OptiMind: Improving Small Language Models for Optimization Modeling through Formulation Guidance and Multi-Turn Learning
- Staying in the Sweet Spot: Boost Reasoning Evolution via Responsive Rollout Difficulty Manipulation
- Examining the Impact of Data and Training Methods on Emergent and Subliminal Misalignment
- Rethinking Black-Box On-Policy Distillation: A Rubric-based Analysis