Reducing Batch Makespan for LLM Agent Workloads through Inference-Environment Coordination
cs.DC, cs.AI
Submitted: 2026-11-12
Updated: 2026-11-12
Code: https://github.com/scaleapi/mcp-atlas
Terminology
Sources
- MCP-Atlas: A Large-Scale Benchmark for Tool-Use Competency with Real MCP Servers
- SAGA: Workflow-Atomic Scheduling for AI Agent Inference on GPU Clusters
- Kimi K3: Open Frontier Intelligence
- From Tensor Buffer to Distributed Memory Hierarchy: A Survey of KV Cache Management for LLM Serving
- Autellix: An Efficient Serving Engine for LLM Agents as General Programs
- Asymmetric Virtual Memory Paging for Hybrid Mamba-Transformer Inference
- Mooncake: A KVCache-centric Disaggregated Architecture for LLM Serving
- Tutti: Making SSD-Backed KV Cache Practical for Long-Context LLM Serving
- $\tau$-Knowledge: Evaluating Conversational Agents over Unstructured Knowledge
- Preble: Efficient Distributed Prompt Scheduling for LLM Serving
- Efficient LLM Serving for Agentic Workflows: A Data Systems Perspective
- Idleness is Relative: Exploiting Tool-Call Idle Windows for Offloading in Agentic Systems with MORI
- ARL-Tangram: Unleash the Resource Efficiency in Agentic Reinforcement Learning
- PEEK: Predictive Queue-Informed KV Cache Management for LLM Serving
- DualMap: Enabling Both Cache Affinity and Load Balancing for Distributed LLM Serving
- Observation, Not Prediction: Conversation-Level Disaggregated Scheduling for Agentic Serving
- WideSearch: Benchmarking Agentic Broad Info-Seeking
- Consistent Hashing with Bounded Loads
Related papers
- Programmable Probabilistic Computer with 1,000,000 p-bits
- BatchGen: An Architecture for Scalable and Efficient Batch Inference
- Seamless Reconfiguration for DAG-BFT (Technical Report)
- ELDR: Expert-Locality-Aware Decode Routing for PD-Disaggregated MoE Serving
- iScheduler: Reinforcement Learning-Driven Continual Optimization for Large-Scale Resource Investment Problems
- TIRx: A Unified Tile Primitive ML Compiler Across Heterogeneous Backends