HYPIC: Position-Independent Caching for Hybrid-Attention LLM Serving
cs.DC
Submitted: 2026-07-01
Updated: 2026-11-04
Code: https://github.com/ai-dynamo/nixl
Terminology
Sources
- MiniMax-M1: Scaling Test-Time Compute Efficiently with Lightning Attention
- KV Packet: Recomputation-Free Context-Independent KV Caching for LLMs
- Kimi Linear: An Expressive, Efficient Attention Architecture
- Jamba: A Hybrid Transformer-Mamba Language Model
- Lightning Attention-2: A Free Lunch for Handling Unlimited Sequence Lengths in Large Language Models
- Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism
- Retentive Network: A Successor to Transformer for Large Language Models
- Every Attention Matters: An Efficient Hybrid Architecture for Long-Context Reasoning
- MEPIC: Memory Efficient Position Independent Caching for LLM Serving
- ProphetKV: User-Query-Driven Selective Recomputation for Efficient KV Cache Reuse in Retrieval-Augmented Generation
- CacheClip: Accelerating RAG with Effective KV Cache Reuse
- KVShare: An LLM Service System with Efficient and Effective Multi-Tenant KV Cache Reuse
- You Need an Encoder for Native Position-Independent Caching
- $A^3$: Attention-Aware Accurate KV Cache Fusion for Fast Large Language Model Serving
Related papers
- BatchGen: An Architecture for Scalable and Efficient Batch Inference
- Seamless Reconfiguration for DAG-BFT (Technical Report)
- ELDR: Expert-Locality-Aware Decode Routing for PD-Disaggregated MoE Serving
- iScheduler: Reinforcement Learning-Driven Continual Optimization for Large-Scale Resource Investment Problems
- TIRx: A Unified Tile Primitive ML Compiler Across Heterogeneous Backends
- AgentWeave: Efficient Distributed Agent Serving via Flow Decomposition