VibeServe: Can AI Agents Build Bespoke LLM Serving Systems?

arXiv:2605.06068 · cs.AI, cs.DC · Submitted 2026-05-07 · Read on arXiv

cs.AI, cs.DC

Submitted: 2026-05-07

Updated: 2026-10-12

Code: https://github.com/uw-syfi/vibe-serve

Terminology

• AI Agents — 1× in this paper · explained in 2 episodes
• Agentic Loop — 1× in this paper · explained in 1 episode
• Benchmark — 1× in this paper · explained in 2 episodes

Sources

• PerfBench: Can Agents Resolve Real-World Performance Bugs?
• JSONSchemaBench: A Rigorous Benchmark of Structured Outputs for Language Models
• LLM-42: Enabling Determinism in LLM Inference with Verified Speculation
• The Llama 3 Herd of Models
• CodeEditorBench: Evaluating Code Editing Capability of Large Language Models
• EvoEngineer: Mastering Automated CUDA Kernel Code Evolution with Large Language Models
• Glia: A Human-Inspired AI for Automated Systems Design and Optimization
• SWE-Perf: Can Language Models Optimize Code Performance on Real-World Repositories?
• Apple vs. Oranges: Evaluating the Apple Silicon M-Series SoCs for HPC Performance and Efficiency
• SWE-bench: Can Language Models Resolve Real-World GitHub Issues?
• VoxServe: Streaming-Centric Serving System for Speech Language Models
• Improving Coherence and Persistence in Agentic AI for System Optimization
• Moonshine v2: Ergodic Streaming Encoder ASR for Latency-Critical Speech Applications
• XGrammar-2: Dynamic and Efficient Structured Generation Engine for Agentic LLMs
• Jamba: A Hybrid Transformer-Mamba Language Model
• RepoBench: Benchmarking Repository-Level Code Auto-Completion Systems
• Autellix: An Efficient Serving Engine for LLM Agents as General Programs
• SWE-fficiency: Can Language Models Optimize Real-World Repositories on Real Workloads?
• Olmo Hybrid: From Theory to Practice and Back
• AlphaEvolve: A coding agent for scientific and algorithmic discovery

Related papers

• MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
• Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
• Agents' Last Exam
• The Clinician's Veto: Navigating Trust, Liability, and Uncertainty in Autonomous AI Prescribing
• MindHelper: Closed-Loop Embodied Mental-State Reasoning for Precision Intervention
• Incumbent Advantage: Brand Bias and Cognitive Manipulation Dynamics in LLM Recommendation Systems