Rethinking Multi-Image Re-Representation in Multi-Image Understanding
The paper investigates how large multimodal models can better understand multiple images by organizing visual evidence through 're-representation.' It introduces Mosaic, a system allowing models to actively create visual intermediates using image operations like cropping and transformation. Visual re-representation proves most beneficial for tasks needing precise visual evidence, such as hypothesis testing.