EMAR: Efficient Multi-domain Adaptive Retrieval
cs.IR
Submitted: 2026-02-16
Updated: 2026-11-02
Code: https://github.com/pkasela/DRAMA_Domain-Retrieval-using-Adaptive-ModuleAllocation
License: http://creativecommons.org/licenses/by/4.0/
The gist: Neural retrieval systems are increasingly deployed in multi-domain settings, where each domain may have its own vocabulary and word distribution.
Terminology
Abstract
Neural retrieval systems are increasingly deployed in multi-domain settings, where each domain may have its own vocabulary and word distribution. A common approach is to fine-tune separate models for each domain, achieving strong in-domain performance while increasing training costs and storage requirements. In contrast, a single model trained jointly across domains avoids model duplication but weakens robustness under domain shift. This creates a trade-off between effectiveness and resource-efficiency. To address this issue, we propose EMAR, a retrieval system that employs a single frozen model shared across all domains and introduces lightweight domain-specific adapters. At inference time, a learned routing mechanism dynamically selects the most suitable adapter for each query. EMAR enables efficient domain-specific retrieval by sharing the computationally expensive backbone across multiple domains and without maintaining multiple full models. Experimental results on three multi-domain benchmarks spanning seventeen domains and three retrieval architectures show that EMAR achieves retrieval effectiveness on par or even better than fully fine-tuned domain-specific models while requiring storing only a small fraction of parameters per domain. Furthermore, EMAR exhibits strong generalization to previously unseen domains, highlighting its effectiveness as an extensible solution for multi-domain retrieval.
Sources
- Distilling the Knowledge in a Neural Network
- Improving Efficient Neural Ranking Models with Cross-Architecture Knowledge Distillation
- Mixtral of Experts
- How Many Parameters Does Your Task Really Need? Task Specific Pruning with LLM-Sieve
- DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter
- Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer
- Branch-Train-MiX: Mixing Expert LLMs into a Mixture-of-Experts LLM
Related papers
- Revisiting Text Ranking in Deep Research
- GateSID: One Gate to Balance Semantic and Collaborative Signals across the Item Lifecycle
- SAM-D2Q: Aligning Multimodal Doc2Query with Search Demand and Conversion for E-commerce
- GINGER: Spicing Up Graph-Based Vector Search
- Subtraction Gets You More: Gap-Aware Retrieval for Multimodal Multi-Hop QA
- One-Step Retrieval Framework for Real-Time Sponsored Search Ads Using Hierarchical Text Representations