Bootstrapping Audiovisual Speech Recognition in Zero-AV-Resource Scenarios
The study developed a zero-AV-resource framework to improve speech recognition by creating synthetic lip-synchronized videos from static images and real audio. This method allows training for languages lacking native audiovisual data, showing that synthetic visuals provide effective supervision. The resulting model outperforms audio-only baselines and demonstrates better robustness under noise.