Understanding Efficient LLM Serving: An Empirical Study of Frameworks, Optimizations Algorithms, and Real-World Adoption
cs.SE, cs.AI, cs.LG
Submitted: 2026-08-04
Updated: 2026-11-08
Code: https://github.com/InternLM/lmdeploy
Project page: https://nvidia.github.io/TensorRTLLM/overview.html
Terminology
Sources
- A Survey of LLM Inference Systems
- Beyond Efficiency: A Systematic Survey of Resource-Efficient Large Language Models
- Efficient Large Language Models: A Survey
- A Survey of Resource-efficient LLM and Multimodal Foundation Models
- A Survey on Efficient Inference for Large Language Models
- Model Compression and Efficient Inference for Large Language Models: A Survey
- Hybrid LLM: Cost-Efficient and Quality-Aware Query Routing
- Flash-LLM: Enabling Cost-Effective and Highly-Efficient Large Generative Model Inference with Unstructured Sparsity
- AIBrix: Towards Scalable, Cost-Effective Large Language Model Inference Infrastructure
- A Comprehensive Overview of Large Language Models
- Large Language Models: A Survey
- A Survey of Large Language Models for Financial Applications: Progress, Prospects and Challenges
- A Survey on Large Language Model-empowered Autonomous Driving
- Real Faults in Model Context Protocol (MCP) Software: a Comprehensive Taxonomy
- Characterizing Faults in Agentic AI: A Taxonomy of Types, Symptoms, and Root Causes
- The Rise of AI Teammates in Software Engineering (SE) 3.0: How Autonomous Coding Agents Are Reshaping Software Engineering
- Fault Localization in Deep Learning-based Software: A System-level Approach
- Generating GitHub Repository Descriptions: A Comparison of Manual and Automated Approaches
- Prioritizing App Reviews for Developer Responses on Google Play
- LLM-Assisted Topic Reduction for BERTopic on Social Media Data
Related papers
- ArkEval: Benchmarking and Evaluating Automated CodeRepair for ArkTS
- TestPrism: Rethinking Test Evaluation Beyond a Single Reference
- Mining Workflow Graphs for Black-Box Boundary Testing of Conversational LLM Agents
- Falsification-Based Verification of LLM-Generated Optimization Models: Sound Test Batteries and Their Detection Limits
- GitSkills: A Dataset of Agent Skills on GitHub
- For Coding Agents, the Repository Matters