Inverse-LLaVA: Rethinking Multimodal Alignment via Text-to-Vision Mapping
Inverse-LLaVA proposes a new way to connect text and vision by mapping text embeddings directly into a continuous visual space instead of forcing visual features to fit discrete text tokens. This eliminates costly alignment pre-training, allowing for better preservation of visual detail and improved reasoning performance.