Generating Chest X-Ray Counterfactuals by Specialising Foundation Image Models
cs.CV
Submitted: 2026-09-21
Updated: 2026-09-21
Comments: 32 pages, 6 figures, 16 tables. Code: https://github.com/GSK-AI/RadCF/
Code: https://github.com/GSK-AI/RadCF
License: http://creativecommons.org/licenses/by-sa/4.0/
The gist: Counterfactual image generation answers questions about how a subject would have looked under retrospective, hypothetical scenarios.
Terminology
Abstract
Counterfactual image generation answers questions about how a subject would have looked under retrospective, hypothetical scenarios. Recent methods have improved perceptual quality, identity preservation and faithfulness to an underlying causal model, but their adoption in healthcare is limited by scarce annotated data, distribution shift between datasets, and mismatches between pretrained generative models and those required for counterfactual inference. We propose specialisation, a data and parameter-efficient framework for adapting pretrained, non-causal generative models into causal mechanisms under distribution shift. Based on this framework, we train a radiology counterfactual image generation model, called RadCF, using latent flow matching. We validate our approach on three chest X-ray datasets spanning different dataset shifts, data volumes, and counterfactual questions, associated with challenging, highly-localised interventions. Our results show that RadCF and specialisation improve counterfactual soundness over existing methods while being data and parameter efficient, and that the resulting counterfactuals can detect and mitigate shortcut learning in a downstream medical classifier. Code is available at https://github.com/GSK-AI/RadCF/.
Sources
- Very Deep VAEs Generalize Autoregressive Models and Can Outperform Them on Images
- BiomedJourney: Counterfactual Biomedical Image Generation by Instruction-Learning from Multimodal Patient Journeys
- Counterfactual Fairness
- REPA-E: Unlocking VAE for End-to-End Tuning with Latent Diffusion Transformers
- SiT: Exploring Flow and Diffusion-based Generative Models with Scalable Interpolant Transformers
- Benchmarking Counterfactual Image Generation
- Counterfactual Off-Policy Evaluation with Gumbel-Max Structural Causal Models
- DINOv2: Learning Robust Visual Features without Supervision
- Scalable Diffusion Models with Transformers
- Generative AI for Medical Imaging: extending the MONAI Framework
- SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis
- CheXNet: Radiologist-Level Pneumonia Detection on Chest X-Rays with Deep Learning
- Causal-Adapter: Taming Text-to-Image Diffusion for Faithful Counterfactual Generation
- Factored Classifier-Free Guidance
- Representation Alignment for Generation: Training Diffusion Transformers Is Easier Than You Think
Related papers
- Loss Knows Best: Detecting Annotation Errors in Videos via Loss Trajectories
- AnchorWeave: World-Consistent Video Generation with Retrieved Local Spatial Memories
- Benchmarking the Robustness of Foundation Models for Mammography under Domain Shift
- MambaX-Net: Dual-Input Mamba-Enhanced Cross-Attention Network for Longitudinal MRI Segmentation
- TeleOCR: Navigating Document Parsing Across Digital and Camera-Captured Documents
- A Survey on Efficient Vision-Language-Action Models