KVBuffer: IO-aware Serving for Linear Attention
cs.LG, cs.AI
Submitted: 2026-05-18
Updated: 2026-11-17
Terminology
Sources
- SARATHI: Efficient LLM Inference by Piggybacking Decodes with Chunked Prefills
- DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning
- Mamba: Linear-Time Sequence Modeling with Selective State Spaces
- MiniMax-M1: Scaling Test-Time Compute Efficiently with Lightning Attention
- Eagle and Finch: RWKV with Matrix-Valued States and Dynamic Recurrence
- Prefill-as-a-Service: KVCache of Next-Generation Models Could Go Cross-Datacenter
- Retentive Network: A Successor to Transformer for Large Language Models
- STree: Speculative Tree Decoding for Hybrid State-Space Models
- Kimi Linear: An Expressive, Efficient Attention Architecture
Related papers
- A Physics-Regulated Neural Framework for Learning 3D Grain Growth Dynamics
- FlexMoE: Learning One-for-All Multi-Budget Expert Pruning for Flexible MoE Deployment
- OptiMind: Improving Small Language Models for Optimization Modeling through Formulation Guidance and Multi-Turn Learning
- Staying in the Sweet Spot: Boost Reasoning Evolution via Responsive Rollout Difficulty Manipulation
- Examining the Impact of Data and Training Methods on Emergent and Subliminal Misalignment
- Rethinking Black-Box On-Policy Distillation: A Rubric-based Analysis