Seeing is Believing: Benchmarking and Mitigating the Interference of Language Priors on Visual Perception in MLLMs
Language priors in multimodal models can cause them to ignore visual evidence and report what they believe instead of what they see. The study introduces ViSiP-Bench to test this dependency across six prior categories. A training method called MAPS was developed to soften these excessive priors by fine-tuning models on rare, real scenarios, showing it improves prior adherence without sacrificing general performance.