Tevatron Meets Megatron: Expert-Parallel LLM Reranker Training on an Academic Budget
cs.IR, cs.LG
Submitted: 2026-08-02
Updated: 2026-11-08
Code: https://github.com/deepspeedai/DeepSpeed
Terminology
Sources
- Distilling the Knowledge in a Neural Network
- Improving Efficient Neural Ranking Models with Cross-Architecture Knowledge Distillation
- LoRA: Low-Rank Adaptation of Large Language Models
- SPLADE-v3: New baselines for SPLADE
- Training Sparse Mixture Of Experts Text Embedding Models
- Qwen3 Technical Report
- Qwen3 Embedding: Advancing Text Embedding and Reranking Through Foundation Models
- PyTorch FSDP: Experiences on Scaling Fully Sharded Data Parallel
Related papers
- UniRank: Benchmarking Ranking Models for Unified Sequential Modeling and Feature Interaction
- Revisiting Text Ranking in Deep Research
- GateSID: One Gate to Balance Semantic and Collaborative Signals across the Item Lifecycle
- SAM-D2Q: Aligning Multimodal Doc2Query with Search Demand and Conversion for E-commerce
- GINGER: Spicing Up Graph-Based Vector Search
- Subtraction Gets You More: Gap-Aware Retrieval for Multimodal Multi-Hop QA