Decipher-MR: A Vision-Language Foundation Model for 3D MRI Representations
cs.CV, cs.AI, cs.LG
Submitted: 2025-09-25
Updated: 2026-02-03
DOI: 10.1038/s41746-026-02596-4
License: http://creativecommons.org/licenses/by-nc-sa/4.0/
The gist: Magnetic Resonance Imaging is a critical imaging modality in clinical diagnosis and research, yet its complexity and heterogeneity hinder scalable, generalizable machine learning.
Terminology
Abstract
Magnetic Resonance Imaging is a critical imaging modality in clinical diagnosis and research, yet its complexity and heterogeneity hinder scalable, generalizable machine learning. Although foundation models have revolutionized language and vision tasks, their application to MRI remains constrained by data scarcity and narrow anatomical focus. We present Decipher-MR, a 3D MRI-specific vision-language foundation model trained on 200,000 MRI series from over 22,000 studies spanning diverse anatomical regions, sequences, and pathologies. Decipher-MR integrates self-supervised vision learning with report-guided text supervision to build robust representations for broad applications. To enable efficient use, Decipher-MR supports a modular design that enables tuning of lightweight, task-specific decoders attached to a frozen pretrained encoder. Following this setting, we evaluate Decipher-MR across disease classification, demographic prediction, anatomical localization, and cross-modal retrieval, demonstrating consistent improvements over existing foundation models and task-specific approaches. These results position Decipher-MR as a versatile foundation for MRI-based AI in clinical and research settings.
Sources
- GPT-4 Technical Report
- Advancing human-centric AI for robust X-ray analysis through holistic self-supervised learning
- Merlin: A Computed Tomography Vision-Language Foundation Model and Dataset
- Advancing Multimodal Medical Capabilities of Gemini
- MedImageInsight: An Open-Source Embedding Model for General Domain Medical Imaging
- BrainSegFounder: Towards 3D Foundation Models for Neuroimage Segmentation
- Triad: Vision Foundation Model for 3D Magnetic Resonance Imaging
- SAM-Med3D: Towards General-purpose Segmentation Models for Volumetric Medical Images
- The Llama 3 Herd of Models
- How to build the best medical image segmentation algorithm using foundation models: a comprehensive empirical study with Segment Anything Model
Related papers
- TEA: Text Encoder Alignment for Robust Concept Erasure in Text-to-Image Models
- Do Egocentric Video-Language Models Capture Both Hand- and Object-Centric Cues?
- Aura: Consistent Multi-Subject Video Generation via VLM-Grounded Semantic Alignment
- PVRF: All-in-one Adverse Weather Removal via Prior-modulated and Velocity-constrained Rectified Flow
- DLWM: Diverse Latent World Models for Efficient Multimodal Reasoning
- Out of Sight, Still in Mind: Token Compression for Omni-LLMs