RTP-LLM: Building an Evolvable and Reliable LLM Inference Engine at Alibaba Scale
cs.OS
Submitted: 2026-10-31
Updated: 2026-10-31
Code: https://github.com/alibaba/rtp-llm
Terminology
Sources
- Qwen Technical Report
- Medusa: Simple LLM Inference Acceleration Framework with Multiple Decoding Heads
- DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Model
- DeepSeek-V3 Technical Report
- DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligence
- GLM-5: from Vibe Coding to Agentic Engineering
- DeepSpeed-FastGen: High-throughput Text Generation for LLMs via MII and DeepSpeed-Inference
- Qwen2.5-Coder Technical Report
- Mistral 7B
- Kimi K3: Open Frontier Intelligence
- GPT-4 Technical Report
- RTP-LLM: High-Performance Alibaba LLM Inference Engine
- Gemma 3 Technical Report
- LLaMA: Open and Efficient Foundation Language Models
- Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolution