Understanding Efficient LLM Serving: An Empirical Study of Frameworks, Optimizations Algorithms, and Real-World Adoption

arXiv:2608.03036 · cs.SE, cs.AI, cs.LG · Submitted 2026-08-04 · Read on arXiv

cs.SE, cs.AI, cs.LG

Submitted: 2026-08-04

Updated: 2026-11-08

Code: https://github.com/InternLM/lmdeploy

Project page: https://nvidia.github.io/TensorRTLLM/overview.html

Terminology

Sources

Related papers