GoodServe: Globally-coordinated Routing of Agentic LLM Inferences for High Goodput

arXiv:2605.16867 · cs.DC · Submitted 2026-05-16 · Read on arXiv

cs.DC

Submitted: 2026-05-16

Updated: 2026-11-02

Code: https://github.com/BerriAI/litellm

Terminology

Sources

Related papers