Federate the Router: Learning Language Model Routers with Sparse and Decentralized Evaluations
The paper introduces a federated learning framework to train language model routers using private, decentralized query-model evaluation data. Instead of centralizing sensitive data, clients collaborate to learn a shared routing policy. This method improves routing accuracy and cost trade-offs by leveraging diverse local distributions while maintaining client privacy.