EchoDistill: Robust Large Audio Language Models via Noisy-to-Clean Self-Distillation
EchoDistill is a training method that makes Large Audio Language Models robust to noise by using self-distillation. It trains a student model on noisy audio while using its clean counterpart as a teacher to align the student's noisy outputs with the semantic preferences of the clean audio. This process corrects semantic drift and hallucinations, leading to significant performance gains.