Direct Preference Optimization for English-Mandarin Code-Switching Speech Recognition in Audio LLMs
The episode discusses a paper on using Direct Preference Optimization (DPO) to improve English-Mandarin code-switching speech recognition in Audio LLMs. The research addresses three failure modes: language omission, translation instead of transcription, and hallucination. DPO trains the model to prefer preserving the actual mixed-language composition, leading to significant reductions in error rates across different models.