What Score Distillation Transfers to Compact Dense Retrievers
cs.IR
Submitted: 2026-07-13
Updated: 2026-11-11
Code: https://github.com/kdubovikov/score-vector-distillation
Terminology
Sources
- CLIMATE-FEVER: A Dataset for Verification of Real-World Climate Claims
- Distilling the Knowledge in a Neural Network
- Improving Efficient Neural Ranking Models with Cross-Architecture Knowledge Distillation
- Dense Passage Retrieval for Open-Domain Question Answering
- EmbedDistill: A Geometric Knowledge Distillation for Information Retrieval
- NV-Embed: Improved Techniques for Training LLMs as Generalist Embedding Models
- Distilling Dense Representations for Ranking using Tightly-Coupled Teachers
- RD-Suite: A Benchmark for Ranking Distillation
- Qwen3 Embedding: Advancing Text Embedding and Reranking Through Foundation Models
- Teaching Dense Retrieval Models to Specialize with Listwise Distillation and LLM Data Augmentation
- BEIR: A Heterogenous Benchmark for Zero-shot Evaluation of Information Retrieval Models
- FEVER: a large-scale dataset for Fact Extraction and VERification
- TREC-COVID: Constructing a Pandemic Information Retrieval Test Collection
- LEAF: Knowledge Distillation of Text Embedding Models with Teacher-Aligned Representations
- Text Embeddings by Weakly-Supervised Contrastive Pre-training
- HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering
Related papers
- Revisiting Text Ranking in Deep Research
- GateSID: One Gate to Balance Semantic and Collaborative Signals across the Item Lifecycle
- SAM-D2Q: Aligning Multimodal Doc2Query with Search Demand and Conversion for E-commerce
- GINGER: Spicing Up Graph-Based Vector Search
- Subtraction Gets You More: Gap-Aware Retrieval for Multimodal Multi-Hop QA
- One-Step Retrieval Framework for Real-Time Sponsored Search Ads Using Hierarchical Text Representations