LLMRouter: Unified Infrastructure for Developing, Evaluating, and Deploying LLM Routers
Tao Feng, Fangxu Yu, Haozhen Zhang, Zhongjie Dai, Liangqi Yuan, Zijie Lei, Weizhi Zhang, Kunlun Zhu, Haodong Yue, Keyang Xuan, Ge Liu, Jiaxuan You
cs.CL
Submitted: 2026-08-07
Updated: 2026-08-10
Code: https://github.com/openclaw/openclaw
License: http://creativecommons.org/licenses/by/4.0/
The gist: No single large language model (LLM) is optimal across all queries and budget constraints, making model routing essential for cost-effective deployment.
Terminology
Abstract
No single large language model (LLM) is optimal across all queries and budget constraints, making model routing essential for cost-effective deployment. Existing routers adopt diverse formulations and implementations, making fair comparison and extension difficult. We present a unified formulation of LLM routing as a sequential decision process characterized by five components: context encoders, model encoders, scoring functions, decision rules, and learning signals, covering single-turn, multi-turn, and personalized routing. Based on this formulation, we develop an automated pipeline for constructing routing supervision and evaluating routers jointly on response quality and inference cost. The resulting benchmark, xRouteBench, spans generic LLM, memory-augmented, vision, time-series, and personalized routing tasks. We further introduce LLMRouter, an open-source modular infrastructure with more than 16 representative routers. Our empirical study shows that learned routers outperform the strongest fixed-model baseline by 14.6% relatively, lightweight routers become more competitive under tight cost constraints, and user-conditioned routing consistently improves personalization.
Sources
- The Llama 4 Herd: Architecture, Training, Evaluation, and Deployment Notes
- gpt-oss-120b & gpt-oss-20b Model Card
- Program Synthesis with Large Language Models
- FrugalGPT: How to Use Large Language Models While Reducing Cost and Improving Performance
- Evaluating Large Language Models Trained on Code
- Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge
- Training Verifiers to Solve Math Word Problems
- Cost-Effective Online Multi-LLM Selection with Versatile Reward Models
- PersonalizedRouter: Personalized LLM Routing via Graph-based User Preference Modeling
- Hybrid LLM: Cost-Efficient and Quality-Aware Query Routing
- GraphRouter: A Graph-based Router for LLM Selections
- GraphPlanner: Graph Memory-Augmented Agentic Routing for Multi-Agent LLMs
- Prompt-to-Leaderboard
- Scaling Synthetic Data Creation with 1,000,000,000 Personas
- The Llama 3 Herd of Models
- Measuring Massive Multitask Language Understanding
- Measuring Mathematical Problem Solving With the MATH Dataset
- RouterBench: A Benchmark for Multi-LLM Routing System
- VL-RouterBench: A Benchmark for Vision-Language Model Routing
- RouterEval: A Comprehensive Benchmark for Routing LLMs to Explore Model-level Scaling Up in LLMs
Related papers
- Exploring Solution Divergence and Its Effect on Large Language Model Problem Solving
- Ishigaki-IDS-Bench: A Benchmark for Generating Information Delivery Specification from BIM Information Requirements
- Subliminal Steering: Stronger Encoding of Hidden Signals
- MedStruct-S: A Benchmark for Key Discovery, Key-Conditioned QA and Semi-Structured Extraction from OCR Clinical Reports
- The End of Transformers? On Challenging Attention and the Rise of Sub-Quadratic Architectures
- Untangling the Mechanisms of Misleading Context in Medical Question Answering