On the Value of Cross-Modal Misalignment in Multimodal Representation Learning
The research investigates cross-modal misalignment in image-text learning using Latent Variable Models to formalize selection and perturbation biases. The study proves that models retain only semantic variables invariant to these biases, suggesting misalignment can act as a beneficial regularizer for robustness.