A Controlled Experimental Evaluation of Configuration and Cache Architecture in LLM Serving Engines
cs.DC, cs.LG, cs.PF
Submitted: 2026-11-04
Updated: 2026-11-04
Code: https://github.com/FaizanAnwar01/llm-serving-config-eval
Terminology
Sources
- TokenCake: A KV-Cache-centric Serving Framework for LLM-based Multi-Agent Applications
- SAGA: Workflow-Atomic Scheduling for AI Agent Inference on GPU Clusters
- KVFlow: Efficient Prefix Caching for Accelerating LLM-Based Multi-Agent Workflows
- Beyond Per-Token Pricing: A Concurrency-Aware Methodology for LLM Infrastructure Cost Estimation
- ServeGen: Workload Characterization and Generation of Large Language Model Serving in Production
Related papers
- Programmable Probabilistic Computer with 1,000,000 p-bits
- BatchGen: An Architecture for Scalable and Efficient Batch Inference
- Seamless Reconfiguration for DAG-BFT (Technical Report)
- ELDR: Expert-Locality-Aware Decode Routing for PD-Disaggregated MoE Serving
- iScheduler: Reinforcement Learning-Driven Continual Optimization for Large-Scale Resource Investment Problems
- TIRx: A Unified Tile Primitive ML Compiler Across Heterogeneous Backends