Compositional Cross-Modality Translation via Whole-Volume Multitask Latent Flow Matching
Daniele Molino, Alessio Zoboli, Camillo Maria Caruso, Valerio Guarrasi, Paolo Soda
cs.CV, cs.AI
Submitted: 2026-08-08
Updated: 2026-08-11
Comments: Accepted ad Sashimi 2026
Code: https://github.com/arco-group/Whole-Volume-Latent-FM
License: http://creativecommons.org/licenses/by-nc-nd/4.0/
The gist: Cross-modality medical image translation can reduce the burden of multi-modal acquisitions, yet the field remains constrained by two coupled limitations: methods operate on 2D slices or 3D patches
Terminology
Abstract
Cross-modality medical image translation can reduce the burden of multi-modal acquisitions, yet the field remains constrained by two coupled limitations: methods operate on 2D slices or 3D patches rather than whole volumes, and train a separate model for each translation task. Both stem from a single cause, the absence of a sufficiently strong volumetric prior, which forces generative models to learn anatomical appearance and cross-modality mapping simultaneously, an ill-posed problem at the scale of available paired datasets. We propose to decouple these objectives. A large-scale pretrained 3D variational autoencoder provides a compact latent representation of volumetric appearance, reducing translation to a conditional flow-matching problem. This compression makes whole-volume processing tractable, while a resolution-aware sampling strategy preserves native anatomical scale. We train a single model jointly across inter-modality (MRI to CT, CBCT to CT) and intra-modality (MRI to MRI) tasks over three multi-center datasets. Across all tasks, whole-volume processing outperforms its patch-based counterpart, and the multi-task model matches task-specific baselines while replacing N networks with one. Crucially, joint training unlocks capabilities inaccessible to task-specific approaches: zero-shot generalization to anatomical regions unseen during training, within 0.15 SSIM of the fully supervised model, and compositional cross-dataset translation along paths never directly supervised. These results suggest that combining a strong volumetric prior with multitask training is a scalable route toward synthesis systems that generalize beyond their training distribution. Code is available at https://github.com/arco-group/Whole-Volume-Latent-FM.
Sources
- The RSNA-ASNR-MICCAI BraTS 2021 Benchmark on Brain Tumor Segmentation and Radiogenomic Classification
- Machine Learning with Multi-Site Imaging Data: An Empirical Study on the Impact of Scanner Effects
- Sample-Aware Test-Time Adaptation for Medical Image-to-Image Translation
- Diffusion Models for Medical Image Analysis: A Comprehensive Survey
- Flow Matching for Generative Modeling
- From Alignment to Synthesis: Contrastive Volumetric Grounding for Text-to-CT Generation
- Retrieval-Augmented Anatomical Guidance for Text-to-CT Generation
- Cross Modality Image Translation In Medical Imaging Using Generative Frameworks
Related papers
- Loss Knows Best: Detecting Annotation Errors in Videos via Loss Trajectories
- AnchorWeave: World-Consistent Video Generation with Retrieved Local Spatial Memories
- Benchmarking the Robustness of Foundation Models for Mammography under Domain Shift
- MambaX-Net: Dual-Input Mamba-Enhanced Cross-Attention Network for Longitudinal MRI Segmentation
- TeleOCR: Navigating Document Parsing Across Digital and Camera-Captured Documents
- A Survey on Efficient Vision-Language-Action Models