DINOspec: Efficient Multimodal Alignment of Vision and Spectral Foundation Models for Astronomy

Name: DINOspec: Efficient Multimodal Alignment of Vision and Spectral Foundation Models for Astronomy
Uploaded: 2026-11-01T04:26:35.434241+00:00
Description: DINOspec combines pre-trained vision and spectral models from separate sources by using lightweight adapters to align their representations. This method improves galaxy morphology classification (F1: 0.72->0.78) while reusing existing foundation models, showing alignment is most effective when modalities offer complementary information.

DINOspec combines pre-trained vision and spectral models from separate sources by using lightweight adapters to align their representations. This method improves galaxy morphology classification (F1: 0.72->0.78) while reusing existing foundation models, showing alignment is most effective when modalities offer complementary information.

Open the full episode