Label-Preserving Fourier Domain Adaptation with Generative-AI and Real Style Donors for 3D Mitral Valve Segmentation
Listen
Radio episode about this paper
Transcript
Introduction to the show: ident: Electrical Engineering. The day in electrical engineering: signals, systems and the hardware that carries them.
Rosa: We're looking at the paper: "Label-Preserving Fourier Domain Adaptation with Generative-AI and Real Style Donors for 3D Mitral Valve Segmentation".
Dev: Here's the gist of the paper. Accurate segmentation of the mitral valve from three-dimensional transesophageal echocardiography (3D TEE) is severely restricted by the scarcity of publicly available expert annotated volumetric datasets.
Rosa: This study investigates Fourier Domain Adaptation (FDA), utilizing general-purpose generative-AI images as annotation-free style donors for 3D mitral valve segmentation.
Dev: The Core Problem and Proposed Solution Accurate segmentation of the mitral valve from three-dimensional transesophageal echocardiography (3D TEE) is severely restricted by the scarcity of publicly available expert annotated volumetric datasets.
Rosa: Manual segmentation of a single 3D TEE volume can take a trained operator upwards of two hours Deep learning has substantially reduced this burden with models like DeepMitral, which segments the mitral leaflets directly from raw TEE volumes in seconds rather than hours.
Title and authors: Rosa: So the paper we're looking at is titled "Label-Preserving Fourier Domain Adaptation with Generative-AI and Real Style Donors for three dee Mitral Valve Segmentation <ref:2611.05106#pg1,Label-Preserving Fourier Domain Adaptation with Generative-AI and Real Style Donors>." I think that title itself tells you a lot about what they're trying to do. It’s taking this problem of not having enough labeled data from three dimensional transesophageal echocardiography, or TEE scans, and trying to solve it by using some kind of Fourier Domain Adaptation with images from generative AI as the style reference.
Dev: Yeah, it sounds like they are tackling that scarcity issue directly. The core idea is using those general purpose generative AI images—things like ChatGPT or Gemini—not to create new segmentations, but just to provide a style for the segmentation model when it’s looking at real TEE volumes.
Taro: So instead of needing someone to painstakingly draw masks on every single scan, this method lets the system learn how to adapt its appearance based on these generated images. It's turning an annotation problem into something they call a label preserving appearance transfer problem.
Rosa: Exactly, and that means they aren't trying to generate fake three dee volumes or new ground truth masks from scratch, which is usually a much harder thing to get right <ref:2611.05106#pg1>. They are using the real expert masks as the foundation for this process.
Dev: And they are specifically focusing on transferring those low frequency amplitude components onto a real, already annotated three dimensional TEE volume by applying Fourier Domain Adaptation slice wise and retaining the phase spectrum of that real volume. That’s a specific technical way to bridge that gap between two dimensions, 2D images and three dee scans <ref:2611.05106#pg2>.
The paper's summary: Rosa: So what they actually did in the paper is set up this whole system to test several conditions against each other. They used a DeepMitral residual U-Net model on the public MVSeg2023 dataset, and they compared six different experimental setups over three runs.
Dev: Those six conditions are real-only training, traditional augmentation, then four different FDA conditions using ChatGPT, Gemini, and Microsoft MAI images as donors. They also included a non AI FDA control using real image style donors.
Taro: And they made the set bigger by taking those forty adapted volumes from each generator and combining them with the one hundred five real training volumes to make a total of one hundred forty five volumes for training, which is more data than just the original ten five annotated training sets.
Rosa: That way, they’re essentially creating additional labeled three dimensional training volumes without needing any new manual segmentation from human experts. That’s their main contribution in terms of how they build the dataset size.
Dev: But the results show that while boundary and surface related errors dropped significantly across all those four FDA conditions compared to the real only baseline, the generalized dice score, or GDS, had some generator dependence. For example, MAI with its FDA method showed a numerically highest mean GDS at zero point eight three zero nine plus or minus zero point one hundred twenty-nine.
Taro: And Gemini's FDA didn't show any significant difference in the GDS compared to the primary baseline, which is a bit surprising if you’re expecting every generator to be equally good at this style transfer.
The paper's improvements: Rosa: The paper highlights a few key things they improved or found interesting about this process. One big improvement is that they managed to significantly reduce the boundary and surface related errors across all three of the generative AI donor conditions compared to just using real training data alone.
Dev: That reduction in error metrics is definitely where the practical benefit lies for anyone trying to get better segmentation results without having access to huge labeled datasets. But they also found that improving overlap, measured by the GDS, wasn't consistent across all generators; MAI FDA was numerically highest, but Gemini FDA didn't show a significant GDS difference from the baseline.
Taro: It seems like the main takeaway is that Fourier Domain Adaptation remains an important augmentation mechanism because even when comparing it to a non AI control using real image style donors, the performance was comparable to just using the real-only training data.
Rosa: So they are saying that generative AI donors offer a practical means of acquiring appearance diversity for training without needing new annotations for those synthetic three dee masks <ref:2611.05106#pg1>. It’s about getting that extra variation in the model's appearance.
Dev: The authors also noted some limitations, which is important because they aren't claiming perfection. They mentioned that the generalized dice score effect was generator dependent, and that they did a beta sensitivity analysis on the FDA parameter, which showed substantial between run variability and didn't support a universal claim about what the optimal value of that parameter should be.
Conclusion: Rosa: So to wrap up this paper on "Label-Preserving Fourier Domain Adaptation with Generative-AI and Real Style Donors for three dee Mitral Valve Segmentation," the main implication is that Fourier Domain Adaptation can be used to transfer low frequency amplitude components onto real volumes using generative AI images as style donors <ref:2611.05106#pg1,Label-Preserving Fourier Domain Adaptation with Generative-AI and Real Style Donors>.
Dev: They showed that this procedure turns the problem into a label preserving appearance transfer problem, which allows them to create more labeled training volumes without doing any new manual segmentation work. They tested six conditions including traditional augmentation and those four FDA conditions.
Taro: And they found that while boundary and surface errors were reduced across the board, improving overlap wasn't universally shared by the generators; MAI FDA showed a numerically highest mean GDS of zero point eight three zero nine plus or minus zero point one hundred twenty-nine compared to the primary baseline.
Rosa: So for someone listening who just wants to know what this means for them, it suggests that even if you don't have millions of annotated scans, you can still use annotation free images from generative AI as a practical way to add diversity to your training data.
Dev: But remember the caveat they mentioned: the FDA parameter sensitivity analysis showed substantial between run variability, so you can't just pick one setting and expect perfect results every time. They also pointed out that external validation on an independent three dimensional TEE dataset is needed because their test cohort only had forty real MVSeg2023 volumes.
Taro: Exactly, the paper suggests that generative AI donor content isn't uniquely superior to real image donors, since the results for real training-image style donors yielded GDS and HD95 similar to ChatGPT FDA. So FDA is an important mechanism for getting annotation free 2D appearance variation without needing new annotations <ref:2611.05106#pg2>.
Javeria Javed, Ahmed Salim, Muhammad Mahboob Ur Rahman
College of Electrical and Mechanical Engineering, NUST · Electrical Engineering Department, Namal University · Electrical Engineering Department, Information Technology University
eess.IV, cs.CV
Submitted: 2026-11-03
Updated: 2026-11-03
Comments: 7 pages, 1 figure, 3 tables, submitted to IEEE for possible publication
License: http://creativecommons.org/licenses/by/4.0/
Importance score: 80/100
The gist: Accurate segmentation of the mitral valve from three-dimensional transesophageal echocardiography (3D TEE) is severely restricted by the scarcity of publicly available expert annotated volumetric…
Key concepts
- Fourier Domain Adaptation (FDA)
- A method that adapts a target image by transferring specific frequency components (like low-frequency amplitude) from a style image to the target. In this study, it allows 2D image features to be transferred onto 3D TEE slices while preserving the original slice's phase information, effectively adding appearance diversity.
- Generative-AI Style Donors
- Using images generated by models like ChatGPT or Gemini as 'style donors' for training. These AI-generated 2D images provide a practical way to introduce diverse visual appearances into the segmentation training process without requiring new, expensive manual annotations for synthetic 3D masks.
- Slice-wise 2D FDA
- Applying the adaptation technique independently to every slice of a real TEE volume. This bridges the dimensional gap by replacing only the central low-frequency amplitude spectrum of each slice with components from the donor image, ensuring that original phase information is retained for accurate 3D reconstruction.
- Generalized Dice Score (GDS)
- A metric used to evaluate segmentation accuracy, measuring both overlap and boundary precision. The study found that while boundary and surface errors improved across all conditions, the GDS gains were generator-dependent, with MAI-FDA showing the highest mean score.
Terminology
Summary
Accurate segmentation of the mitral valve from three-dimensional transesophageal echocardiography (3D TEE) is severely restricted by the scarcity of publicly available expert annotated volumetric datasets. p. 0 1
Label-Preserving Fourier Domain Adaptation with
Generative-AI and Real Style Donors for 3D Mitral
Valve Segmentation
Javeria Javed, Ahmed Salim, Muhammad Mahboob Ur Rahman Senior Member, IEEE
Abstract—Accurate segmentation of the mitral valve from
three-dimensional transesophageal echocardiography (3D TEE)
is severely restricted by the scarcity of publicly available expertannotated volumetric datasets. To address this critical limitation,
we investigated Fourier Domain Adaptation (FDA), utilizing
general-purpose generative-AI images (from platforms like ChatGPT, Google Gemini, and Microsoft MAI) as annotation-free
style donors for 3D mitral valve segmentation. 1
Label-Preserving Fourier Domain Adaptation with
Generative-AI and Real Style Donors for 3D Mitral
Valve Segmentation
Javeria Javed, Ahmed Salim, Muhammad Mahboob Ur Rahman Senior Member, IEEE
Abstract—Accurate segmentation of the mitral valve from
three-dimensional transesophageal echocardiography (3D TEE)
is severely restricted by the scarcity of publicly available expertannotated volumetric datasets. To address this critical limitation,
we investigated Fourier Domain Adaptation (FDA), utilizing
general-purpose generative-AI images (from platforms like ChatGPT, Google Gemini, and Microsoft MAI) as annotation-free
style donors for 3D mitral valve segmentation.Where the paper says it
Where the paper says it
The Core Problem and Proposed Solution
Accurate segmentation of the mitral valve from three-dimensional transesophageal echocardiography (3D TEE) is severely restricted by the scarcity of publicly available expert annotated volumetric datasets. Manual segmentation of a single 3D TEE volume can take a trained operator upwards of two hours Deep learning has substantially reduced this burden with models like DeepMitral, which segments the mitral leaflets directly from raw TEE volumes in seconds rather than hours. p. 0 model from a 3D TEE volume is essential for pre-operative
planning, in-silico repair simulation, and quantitative assessment of leaflet coaptation, but manual segmentation of a single
3D TEE volume can take a trained operator upwards of two
hours [4], making it impractical for routine clinical workflows. Deep learning has substantially reduced this burden. Despite this progress, the
fundamental data bottleneck of medical image segmentation
persists: MVSeg2023 provides only 105 annotated training
volumes, an order of magnitude smaller than the datasets
that power state-of-the-art segmentation frameworks such as
nnU-Net in other anatomical domains [6]. Because 3D TEE
annotation requires simultaneous expertise in cardiac anatomy
and ultrasound interpretation, this scarcity is unlikely to be
resolved through additional manual labelling alone. We address both obstacles with a single procedure. Rather
than treating synthetic images as independent training examples requiring annotation, we use each 2D image purely
as a style reference and apply Fourier Domain Adaptation
(FDA) [12] to transfer selected low-frequency amplitude components onto a real, already-annotated 3D TEE volume. Because FDA retains the real-image phase, the volume’s spatial
organization is expected to remain largely unchanged, so we
reuse the original expert mask rather than claiming that every
anatomical feature is exactly preserved. This turns the 2Dimage-to-3D-mask annotation problem into a label-preserving
appearance-transfer problem, yielding additional labelled 3D
training volumes without new manual segmentation.Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
How the Training Signal is Built
The methodology involves constructing three generator-guided FDA training sets derived from ChatGPT (GPT-Image), Google Gemini (Nano Banana), and Microsoft MAI-Image-2.6. For each generator, 40 synthetic 2D images of the midesophageal TEE mitral valve view are generated using structured prompts that specify anatomical details and pathology. p. 2 Synthetic Image Generation
For each of three generators, ChatGPT (GPT-Image),
Google Gemini (Nano Banana), and Microsoft MAI-Image2. 6, we generated 40 synthetic 2D images of the midesophageal TEE mitral valve view. Because the
real phase is retained, spatial organization is expected to
remain largely intact, so Mr was reused unchanged: the donor
contributes low-frequency appearance variation while supervision is inherited from the real volume. This lets generated
2D images add appearance diversity to 3D training without
synthetic 3D masks or additional annotation.Where the paper says it
Where the paper says it
Where the Gains Come From
The results show that while boundary and surface error metrics were significantly reduced for all three generator-guided conditions relative to the real-only baseline, the Generalized Dice Score (GDS) effect was generator-dependent. Specifically, MAI-FDA had the numerically highest mean GDS (0. p. 3 The four FDA-based conditions (ChatGPT-
, Gemini-, MAI-, and Non-AI FDA) substantially reduced
mean HD95 and Surface Distance relative to the real-only
baseline, while their overlap results differed. MAI-FDA had
the numerically highest mean GDS (0. ChatGPT-FDA showed a significant GDS
advantage over the optimization-matched control but not over
the primary baseline, and Gemini-FDA showed no significant
GDS difference. Notably, only MAI-FDA provided a statistically
robust and significant increase in GDS compared to the primary
baseline.Where the paper says it
Where the paper says it
Limitations and Future Directions
The study notes several limitations, including that MVSeg2023 is anonymized, meaning independence across cases is assumed rather than verified. Furthermore, only three runs per condition limits inference on between-run variability. p. 5 Second, FDA alters
appearance statistics but creates no new valve anatomy or
pathology. Third, only three runs per condition limit inference
on between-run variability. In other words, generative-AI donor may
be particularly useful when additional real appearance donors
are unavailable or costly to collect. Finally, The FDA parameter β sensitivity analysis therefore
points to substantial between-run variability and does not
support a universal claim that one tested β value is optimal. Also, the test cohort has only 40
real MVSeg2023 volumes. Thus, external validation on an
independent 3D TEE dataset is needed. Real training-image style donors yielded GDS and HD95
similar to ChatGPT-FDA (no significant difference), so FDA
is an important mechanism but generative-AI donor content is
not shown to be uniquely beneficial. Its practical value is as an
annotation-free source of 2D appearance variation transferable
to labelled real volumes, rather than a generator of valid 3D
volumes or labels. In other words, generative-AI donor may
be particularly useful when additional real appearance donors
are unavailable or costly to collect. Finally, The FDA parameter β sensitivity analysis therefore
points to substantial between-run variability and does not
support a universal claim that one tested β value is optimal.Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Conclusion
We evaluated general-purpose generative-AI images as 2D appearance donors for label-preserving, slice-wise Fourier Domain Adaptation in 3D mitral-valve segmentation. In the standardized six-condition three-run comparison, MAI-FDA achieved the numerically highest mean GDS (0. p. 5 CONCLUSION
We evaluated general-purpose generative-AI images as 2D
appearance donors for label-preserving, slice-wise Fourier
Domain Adaptation in 3D mitral-valve segmentation. In the
standardized six-condition three-run comparison, MAI-FDA
achieved the numerically highest mean GDS (0. 00123), whereas
HD95 and Surface Distance were significantly reduced for
all three generator-guided conditions. The most reproducible
effect is thus reduced boundary and surface error, while
improved overlap is not shared across generators. 00123), whereas
HD95 and Surface Distance were significantly reduced for
all three generator-guided conditions. The most reproducible
effect is thus reduced boundary and surface error, while
improved overlap is not shared across generators. Since a non-AI FDA control
using real training images as style donors performed comparably on GDS and HD95, this supports FDA as an important
augmentation mechanism without establishing that generativeAI donors are superior to real-image donors. Generative models nonetheless offer a practical, annotation-free source of
2D appearance diversity that can be transferred to alreadylabelled real 3D volumes without requiring synthetic voxellevel masks.Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.
Improvements for AI systems
-
Fourier Domain Adaptation can be used to transfer
low-frequency amplitude components onto a real, already-annotated 3D TEE volume
using slice-wise 2D FDA to bridge the dimensional mismatch between 2D generated images and 3D segmentation training. This allows the system to acquireadditional labelled 3D training volumes without new manual segmentation.
-
The system can be configured with six experimental conditions, including
real-only training, traditional augmentation, FDA with each generative source, and a non-AI FDA control using real-image style donors.
This provides a rigorous framework for comparing the impact of different style donors. -
The model can be trained on 145 volumes formed by combining 40 adapted volumes with 105 real training volumes, resulting in
additional labelled 3D training volumes without new manual segmentation.
-
The system can be evaluated using metrics like the Generalized Dice Score (GDS), the
95th-percentile Hausdorff Distance (HD95) for boundary discrepancy,
andSurface Distance for mean surface error.
These metrics allow for a comprehensive assessment of both overlap and boundary accuracy. -
By using generative-AI images as style donors, the system can acquire
practical means of acquiring style diversity without new annotations
to augment small medical imaging datasets like MVSeg2023. -
The system can be designed to compare the performance of different generative AI sources, as demonstrated by showing that
only MAI-FDA provided a statistically robust and significant increase in GDS compared to the primary baseline.
Abstract
Accurate segmentation of the mitral valve from three-dimensional transesophageal echocardiography (3D TEE) is severely restricted by the scarcity of publicly available expert- annotated volumetric datasets. To address this critical limitation, we investigated Fourier Domain Adaptation (FDA), utilizing general-purpose generative-AI images (from platforms like Chat- GPT, Google Gemini, and Microsoft MAI) as annotation-free style donors for 3D mitral valve segmentation. We employed a DeepMitral residual U-Net model on the public MVSeg2023 dataset, comparing six conditions over three runs: real-only training, traditional augmentation, FDA with each generative source, and a non-AI FDA control using real-image style donors, plus an optimization-matched real-only control and a sensitivity analysis. Case-level paired testing showed that FDA with these generative sources led to significant reductions in boundary- and surface-related errors across all three donor conditions relative to the primary real-only baseline. However, the impact on the core segmentation metric, Generalized Dice Score (GDS), proved generator-dependent. ChatGPT-FDA showed a significant GDS advantage over the optimization-matched control but not over the primary baseline, and Gemini-FDA showed no significant GDS difference. Notably, only MAI-FDA provided a statistically robust and significant increase in GDS compared to the primary baseline. The sensitivity was not robust across training runs. These findings confirm that FDA remains an important augmentation mechanism. While generative-AI images offer a practical means of acquiring style diversity without new annotations, the non-AI control performed comparably to ChatGPT-FDA, so we do not conclude that this approach is superior to training with real-image donors.
Sources
- Recognizing Pneumonia in Real-World Chest X-rays with a Classifier Trained with Images Synthetically Generated by Nano Banana
- MONAI: An open-source framework for deep learning in healthcare
- Improving Performance, Robustness, and Fairness of Radiographic AI Models with Finely-Controllable Synthetic Data
Related papers
- HDNet: High-resolution Dual-domain Learning for Spectral Compressive Imaging
- Mask-guided Spectral-wise Transformer for Efficient Hyperspectral Image Reconstruction
- FDIR: Fidelity-Anchored Decompressed Image Restoration for Human and Machine Perception
- A Disease-Centric Vision-Language Foundation Model for Precision Oncology in Kidney Cancer
- CADS: A Comprehensive Anatomical Dataset and Segmentation for Whole-Body Anatomy in Computed Tomography
- Task-Aware Bitrate Thresholds for Codec-Robust Ultrasound Segmentation