Efficient Audiovisual Speech Processing via MUTUD: Multimodal Training and Unimodal Deployment
The episode discusses Honglie Chen et al.'s paper proposing the Multimodal Training and Unimodal Deployment (MUTUD) framework for efficient audiovisual speech processing. The framework allows a model to be trained using all modalities but deployed using only one, leveraging a Temporally Aligned Modality feature Estimation module to predict missing data. The authors focus on achieving high performance while significantly reducing model size and computational overhead for deployment.