CARD: Cross-component Audio Representation Distillation for Encoder-Free Audio Captioning
CARD creates an encoder-free audio captioning model by removing the audio encoder at inference using teacher-guided distillation. It trains a student model with a projector and a language model by guiding them with knowledge from a frozen CLAP encoder across different parts of the architecture. The key finding is that placing teacher knowledge strategically, matching lower acoustic cues to the projector and higher abstract cues to the language model, significantly boosts performance over simpler distillation methods.