TaxDistill: Improving Metagenomic Taxonomic Annotation via Distilled Genomic Foundation Models
cs.LG, cs.AI
Submitted: 2026-05-22
Updated: 2026-09-30
Comments: The manuscript contains 12 pages, 5 figures, and 4 tables
License: http://creativecommons.org/licenses/by/4.0/
The gist: Metagenomic taxonomic annotation is essential for interpreting complex microbial communities, yet reliable annotation remains challenging under reference database incompleteness and ambiguous
Terminology
Abstract
Metagenomic taxonomic annotation is essential for interpreting complex microbial communities, yet reliable annotation remains challenging under reference database incompleteness and ambiguous taxonomic boundaries. Existing similarity-based tools are efficient, but they often produce noisy pseudo-labels in complex environments; learning-based post-hoc correction methods can further inherit this noise when trained on hard pseudo-labels generated by upstream classifiers. We propose TaxDistill, a plug-and-play knowledge distillation framework for reliable metagenomic taxonomic recalibration. TaxDistill uses the genomic foundation model GenomeOcean as a semantic teacher and performs one-way online distillation, where the teacher is updated by hierarchical taxonomic supervision while its detached soft probability distributions guide a lightweight student network. This design mitigates the student's overfitting to noisy hard pseudo-labels and supports confidence based rejection of potentially unreliable predictions. Comprehensive experiments on seven diverse CAMI2 datasets with MMseqs2, Metabuli, and Kraken2 show that TaxDistill improves annotation reliability over existing baselines in most scenarios. For example, on the Gastrointestinal dataset, TaxDistill improves the F1 score of MMseqs2 from 0.763 to 0.941, surpassing the Taxometer baseline. Our results suggest that large genomic foundation models can provide semantic supervision for noisy scientific annotations, enabling lightweight post-hoc recalibration of existing metagenomic annotation results.
Sources
- Distilling the Knowledge in Data Pruning
- Distilling the Knowledge in a Neural Network
- Strong Teacher Not Needed? On Distillation in LLM Pretraining
- On-the-Fly Adaptation to Quantization: Configuration-Aware LoRA for Efficient Fine-Tuning of Quantized LLMs
- Understanding deep learning requires rethinking generalization
Related papers
- Polynomial-Augmented Neural Networks (PANNs) with Weak Orthogonality Constraints for Enhanced Function and PDE Approximation
- AIRL-S: Unifying Reinforcement Learning and Search-Based Test-Time Scaling via Adversarial Inverse Reinforcement Learning
- Transformers as Bayesian In-Context Experimenters: Smoothness-Adaptive Efficient ATE Estimation
- Convergence issues in Relational Concept Analysis based on AOC-posets
- Beliefs Beyond Posteriors: The Predictive Cost of ELBO Binding
- Understanding Diffusion Models via Ratio-Based Function Approximation with SignReLU Networks