ROCm InferaSim: Cut LLM Serving Trial Costs and Deploy the Best Configuration
cs.AI, cs.DC
Submitted: 2026-11-02
Updated: 2026-11-02
Code: https://github.com/ROCm/rccl
Terminology
Sources
- SPEED-Bench: A Unified and Diverse Benchmark for Speculative Decoding
- SARATHI: Efficient LLM Inference by Piggybacking Decodes with Chunked Prefills
- Demystifying AI Platform Design for Distributed Inference of Next-Generation LLM models
- DSpark: Confidence-Scheduled Speculative Decoding with Semi-Autoregressive Generation
- DeepSeek-V3 Technical Report
- DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligence
- Frontier: Towards Comprehensive and Accurate LLM Inference Simulation
- GLM-5: from Vibe Coding to Agentic Engineering
- The Llama 3 Herd of Models
- BestServe: Serving Strategies with Optimal Goodput in Collocation and Disaggregation Architectures
- Kimi K3: Open Frontier Intelligence
- EAGLE-3: Scaling up Inference Acceleration of Large Language Models via Training-Time Test
- APEX: An Extensible and Dynamism-Aware Simulator for Automated Parallel Execution in LLM Serving
- LMCache: An Efficient KV Cache Layer for Enterprise-Scale LLM Inference
- gpt-oss-120b & gpt-oss-20b Model Card
- Prefill-as-a-Service: KVCache of Next-Generation Models Could Go Cross-Datacenter
- Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism
- tritonBLAS: Triton-based Analytical Approach for GEMM Kernel Parameter Selection
- AIConfigurator: Lightning-Fast Configuration Optimization for Multi-Framework LLM Serving
- Qwen3 Technical Report
Related papers
- Multi-Objective Structured Pruning of LLMs for Latency and Model Size Optimization
- MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
- Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
- Embeddings for Preferences, Not Semantics
- Can We Trust Item Response Theory for AI Evaluation?
- Diagnosing Task Insensitivity in Language Agents