AgentServe: Algorithm-System Co-Design for Efficient Agentic AI Serving on a Consumer-Grade GPU
cs.DC
Submitted: 2026-03-11
Updated: 2026-11-05
Code: https://github.com/langchain-ai/langchain
Terminology
Sources
- Small Language Models are the Future of Agentic AI
- Foundation Models for Decision Making: Problems, Methods, and Opportunities
- Qwen2.5-1M Technical Report
- Nexus:Proactive Intra-GPU Disaggregation of Prefill and Decode in LLM Serving
- semi-PD: Towards Efficient LLM Serving via Phase-Wise Disaggregated Computation and Unified Storage
- Towards High-Goodput LLM Serving with Prefill-decode Multiplexing
- The Llama 3 Herd of Models
- Fast Distributed Inference Serving for Large Language Models
- DuoServe-MoE: Dual-Phase Expert Prefetch and Caching for LLM Inference QoS Assurance
- Autellix: An Efficient Serving Engine for LLM Agents as General Programs
Related papers
- BatchGen: An Architecture for Scalable and Efficient Batch Inference
- Seamless Reconfiguration for DAG-BFT (Technical Report)
- ELDR: Expert-Locality-Aware Decode Routing for PD-Disaggregated MoE Serving
- iScheduler: Reinforcement Learning-Driven Continual Optimization for Large-Scale Resource Investment Problems
- TIRx: A Unified Tile Primitive ML Compiler Across Heterogeneous Backends
- AgentWeave: Efficient Distributed Agent Serving via Flow Decomposition