VIDA: A Dataset for Visually Dependent Ambiguity in Multimodal Machine Translation
The research introduces VIDA, a new dataset of 2,500 multimodal translation examples focusing on ambiguities caused by visual input. It proposes Disambiguation-Centric Metrics to specifically measure how well models resolve these visual ambiguities. Experiments show that using Chain-of-Thought Supervised Fine-Tuning (CoT-SFT) significantly improves performance and generalization when translating text based on images, especially for handling complex visual dependencies.