Recompute or Restore? Host-Tier KV for Multi-Model Agent Workflows on a Static-Graph NPU Serving Stack
cs.DC, cs.LG, cs.PF
Submitted: 2026-11-11
Updated: 2026-11-11
Terminology
Sources
- CONCUR: High-Throughput Agentic Batch Inference of LLM via Congestion-Based Concurrency Control
- ThunderAgent: A Simple, Fast and Program-Aware Agentic Inference System
- LMCache: An Efficient KV Cache Layer for Enterprise-Scale LLM Inference
- Autellix: An Efficient Serving Engine for LLM Agents as General Programs
- KVFlow: Efficient Prefix Caching for Accelerating LLM-Based Multi-Agent Workflows
- Qwen3 Technical Report
Related papers
- Programmable Probabilistic Computer with 1,000,000 p-bits
- BatchGen: An Architecture for Scalable and Efficient Batch Inference
- Seamless Reconfiguration for DAG-BFT (Technical Report)
- ELDR: Expert-Locality-Aware Decode Routing for PD-Disaggregated MoE Serving
- iScheduler: Reinforcement Learning-Driven Continual Optimization for Large-Scale Resource Investment Problems
- TIRx: A Unified Tile Primitive ML Compiler Across Heterogeneous Backends