AutoAdapt: Automatic Domain Discovery Enables Low-Cost Extensibility
AutoAdapt automatically discovers different data domains within instruction-tuned models to enable low-cost extensibility. It trains independent Low-Rank Adaptation (LoRA) adapters for each domain in parallel and uses parameter-free routing at inference. This allows adding new domains or updating existing ones without costly full model retraining, offering a modular solution for evolving environments.