Smart Model Routing for Order-of-Magnitude LLM Cost Reduction
cs.DC, cs.LG
Submitted: 2026-11-11
Updated: 2026-11-11
Code: https://github.com/dshah133/llm-routing-formula-validation
Terminology
Sources
- FrugalGPT: How to Use Large Language Models While Reducing Cost and Improving Performance
- Hybrid LLM: Cost-Efficient and Quality-Aware Query Routing
- RouteLLM: Learning to Route LLMs with Preference Data
- RouterBench: A Benchmark for Multi-LLM Routing System
- Large Language Model Routing with Benchmark Datasets
- Doing More with Less: A Survey on Routing Strategies for Resource Optimisation in Large Language Model-Based Systems
- BEST-Route: Adaptive LLM Routing with Test-Time Optimal Compute
- Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer
Related papers
- Programmable Probabilistic Computer with 1,000,000 p-bits
- BatchGen: An Architecture for Scalable and Efficient Batch Inference
- Seamless Reconfiguration for DAG-BFT (Technical Report)
- ELDR: Expert-Locality-Aware Decode Routing for PD-Disaggregated MoE Serving
- iScheduler: Reinforcement Learning-Driven Continual Optimization for Large-Scale Resource Investment Problems
- TIRx: A Unified Tile Primitive ML Compiler Across Heterogeneous Backends