Label-Preserving Fourier Domain Adaptation with Generative-AI and Real Style Donors for 3D Mitral Valve Segmentation

arXiv:2611.05106 · eess.IV, cs.CV · Submitted 2026-11-03 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: Electrical Engineering. The day in electrical engineering: signals, systems and the hardware that carries them.

Rosa: We're looking at the paper: "Label-Preserving Fourier Domain Adaptation with Generative-AI and Real Style Donors for 3D Mitral Valve Segmentation".

Dev: Here's the gist of the paper. Accurate segmentation of the mitral valve from three-dimensional transesophageal echocardiography (3D TEE) is severely restricted by the scarcity of publicly available expert annotated volumetric datasets.

Rosa: This study investigates Fourier Domain Adaptation (FDA), utilizing general-purpose generative-AI images as annotation-free style donors for 3D mitral valve segmentation.

Dev: The Core Problem and Proposed Solution Accurate segmentation of the mitral valve from three-dimensional transesophageal echocardiography (3D TEE) is severely restricted by the scarcity of publicly available expert annotated volumetric datasets.

Rosa: Manual segmentation of a single 3D TEE volume can take a trained operator upwards of two hours Deep learning has substantially reduced this burden with models like DeepMitral, which segments the mitral leaflets directly from raw TEE volumes in seconds rather than hours.

Title and authors: Rosa: So the paper we're looking at is titled "Label-Preserving Fourier Domain Adaptation with Generative-AI and Real Style Donors for three dee Mitral Valve Segmentation <ref:2611.05106#pg1,Label-Preserving Fourier Domain Adaptation with Generative-AI and Real Style Donors>." I think that title itself tells you a lot about what they're trying to do. It’s taking this problem of not having enough labeled data from three dimensional transesophageal echocardiography, or TEE scans, and trying to solve it by using some kind of Fourier Domain Adaptation with images from generative AI as the style reference.

Dev: Yeah, it sounds like they are tackling that scarcity issue directly. The core idea is using those general purpose generative AI images—things like ChatGPT or Gemini—not to create new segmentations, but just to provide a style for the segmentation model when it’s looking at real TEE volumes.

Taro: So instead of needing someone to painstakingly draw masks on every single scan, this method lets the system learn how to adapt its appearance based on these generated images. It's turning an annotation problem into something they call a label preserving appearance transfer problem.

Rosa: Exactly, and that means they aren't trying to generate fake three dee volumes or new ground truth masks from scratch, which is usually a much harder thing to get right <ref:2611.05106#pg1>. They are using the real expert masks as the foundation for this process.

Dev: And they are specifically focusing on transferring those low frequency amplitude components onto a real, already annotated three dimensional TEE volume by applying Fourier Domain Adaptation slice wise and retaining the phase spectrum of that real volume. That’s a specific technical way to bridge that gap between two dimensions, 2D images and three dee scans <ref:2611.05106#pg2>.

The paper's summary: Rosa: So what they actually did in the paper is set up this whole system to test several conditions against each other. They used a DeepMitral residual U-Net model on the public MVSeg2023 dataset, and they compared six different experimental setups over three runs.

Dev: Those six conditions are real-only training, traditional augmentation, then four different FDA conditions using ChatGPT, Gemini, and Microsoft MAI images as donors. They also included a non AI FDA control using real image style donors.

Taro: And they made the set bigger by taking those forty adapted volumes from each generator and combining them with the one hundred five real training volumes to make a total of one hundred forty five volumes for training, which is more data than just the original ten five annotated training sets.

Rosa: That way, they’re essentially creating additional labeled three dimensional training volumes without needing any new manual segmentation from human experts. That’s their main contribution in terms of how they build the dataset size.

Dev: But the results show that while boundary and surface related errors dropped significantly across all those four FDA conditions compared to the real only baseline, the generalized dice score, or GDS, had some generator dependence. For example, MAI with its FDA method showed a numerically highest mean GDS at zero point eight three zero nine plus or minus zero point one hundred twenty-nine.

Taro: And Gemini's FDA didn't show any significant difference in the GDS compared to the primary baseline, which is a bit surprising if you’re expecting every generator to be equally good at this style transfer.

The paper's improvements: Rosa: The paper highlights a few key things they improved or found interesting about this process. One big improvement is that they managed to significantly reduce the boundary and surface related errors across all three of the generative AI donor conditions compared to just using real training data alone.

Dev: That reduction in error metrics is definitely where the practical benefit lies for anyone trying to get better segmentation results without having access to huge labeled datasets. But they also found that improving overlap, measured by the GDS, wasn't consistent across all generators; MAI FDA was numerically highest, but Gemini FDA didn't show a significant GDS difference from the baseline.

Taro: It seems like the main takeaway is that Fourier Domain Adaptation remains an important augmentation mechanism because even when comparing it to a non AI control using real image style donors, the performance was comparable to just using the real-only training data.

Rosa: So they are saying that generative AI donors offer a practical means of acquiring appearance diversity for training without needing new annotations for those synthetic three dee masks <ref:2611.05106#pg1>. It’s about getting that extra variation in the model's appearance.

Dev: The authors also noted some limitations, which is important because they aren't claiming perfection. They mentioned that the generalized dice score effect was generator dependent, and that they did a beta sensitivity analysis on the FDA parameter, which showed substantial between run variability and didn't support a universal claim about what the optimal value of that parameter should be.

Conclusion: Rosa: So to wrap up this paper on "Label-Preserving Fourier Domain Adaptation with Generative-AI and Real Style Donors for three dee Mitral Valve Segmentation," the main implication is that Fourier Domain Adaptation can be used to transfer low frequency amplitude components onto real volumes using generative AI images as style donors <ref:2611.05106#pg1,Label-Preserving Fourier Domain Adaptation with Generative-AI and Real Style Donors>.

Dev: They showed that this procedure turns the problem into a label preserving appearance transfer problem, which allows them to create more labeled training volumes without doing any new manual segmentation work. They tested six conditions including traditional augmentation and those four FDA conditions.

Taro: And they found that while boundary and surface errors were reduced across the board, improving overlap wasn't universally shared by the generators; MAI FDA showed a numerically highest mean GDS of zero point eight three zero nine plus or minus zero point one hundred twenty-nine compared to the primary baseline.

Rosa: So for someone listening who just wants to know what this means for them, it suggests that even if you don't have millions of annotated scans, you can still use annotation free images from generative AI as a practical way to add diversity to your training data.

Dev: But remember the caveat they mentioned: the FDA parameter sensitivity analysis showed substantial between run variability, so you can't just pick one setting and expect perfect results every time. They also pointed out that external validation on an independent three dimensional TEE dataset is needed because their test cohort only had forty real MVSeg2023 volumes.

Taro: Exactly, the paper suggests that generative AI donor content isn't uniquely superior to real image donors, since the results for real training-image style donors yielded GDS and HD95 similar to ChatGPT FDA. So FDA is an important mechanism for getting annotation free 2D appearance variation without needing new annotations <ref:2611.05106#pg2>.

Javeria Javed, Ahmed Salim, Muhammad Mahboob Ur Rahman

College of Electrical and Mechanical Engineering, NUST · Electrical Engineering Department, Namal University · Electrical Engineering Department, Information Technology University

eess.IV, cs.CV

Submitted: 2026-11-03

Updated: 2026-11-03

Comments: 7 pages, 1 figure, 3 tables, submitted to IEEE for possible publication

License: http://creativecommons.org/licenses/by/4.0/

Importance score: 80/100

The gist: Accurate segmentation of the mitral valve from three-dimensional transesophageal echocardiography (3D TEE) is severely restricted by the scarcity of publicly available expert annotated volumetric…

Key concepts

Fourier Domain Adaptation (FDA)
A method that adapts a target image by transferring specific frequency components (like low-frequency amplitude) from a style image to the target. In this study, it allows 2D image features to be transferred onto 3D TEE slices while preserving the original slice's phase information, effectively adding appearance diversity.
Generative-AI Style Donors
Using images generated by models like ChatGPT or Gemini as 'style donors' for training. These AI-generated 2D images provide a practical way to introduce diverse visual appearances into the segmentation training process without requiring new, expensive manual annotations for synthetic 3D masks.
Slice-wise 2D FDA
Applying the adaptation technique independently to every slice of a real TEE volume. This bridges the dimensional gap by replacing only the central low-frequency amplitude spectrum of each slice with components from the donor image, ensuring that original phase information is retained for accurate 3D reconstruction.
Generalized Dice Score (GDS)
A metric used to evaluate segmentation accuracy, measuring both overlap and boundary precision. The study found that while boundary and surface errors improved across all conditions, the GDS gains were generator-dependent, with MAI-FDA showing the highest mean score.

Terminology

Summary

Accurate segmentation of the mitral valve from three-dimensional transesophageal echocardiography (3D TEE) is severely restricted by the scarcity of publicly available expert annotated volumetric datasets. p. 0

Where the paper says it

1 Label-Preserving Fourier Domain Adaptation with Generative-AI and Real Style Donors for 3D Mitral Valve Segmentation Javeria Javed, Ahmed Salim, Muhammad Mahboob Ur Rahman Senior Member, IEEE Abstract—Accurate segmentation of the mitral valve from three-dimensional transesophageal echocardiography (3D TEE) is severely restricted by the scarcity of publicly available expertannotated volumetric datasets. To address this critical limitation, we investigated Fourier Domain Adaptation (FDA), utilizing general-purpose generative-AI images (from platforms like ChatGPT, Google Gemini, and Microsoft MAI) as annotation-free style donors for 3D mitral valve segmentation.

This study investigates Fourier Domain Adaptation (FDA), utilizing general-purpose generative-AI images as annotation-free style donors for 3D mitral valve segmentation. p. 0
Where the paper says it

1 Label-Preserving Fourier Domain Adaptation with Generative-AI and Real Style Donors for 3D Mitral Valve Segmentation Javeria Javed, Ahmed Salim, Muhammad Mahboob Ur Rahman Senior Member, IEEE Abstract—Accurate segmentation of the mitral valve from three-dimensional transesophageal echocardiography (3D TEE) is severely restricted by the scarcity of publicly available expertannotated volumetric datasets. To address this critical limitation, we investigated Fourier Domain Adaptation (FDA), utilizing general-purpose generative-AI images (from platforms like ChatGPT, Google Gemini, and Microsoft MAI) as annotation-free style donors for 3D mitral valve segmentation.

The Core Problem and Proposed Solution

Accurate segmentation of the mitral valve from three-dimensional transesophageal echocardiography (3D TEE) is severely restricted by the scarcity of publicly available expert annotated volumetric datasets. Manual segmentation of a single 3D TEE volume can take a trained operator upwards of two hours Deep learning has substantially reduced this burden with models like DeepMitral, which segments the mitral leaflets directly from raw TEE volumes in seconds rather than hours. p. 0

Where the paper says it

model from a 3D TEE volume is essential for pre-operative planning, in-silico repair simulation, and quantitative assessment of leaflet coaptation, but manual segmentation of a single 3D TEE volume can take a trained operator upwards of two hours [4], making it impractical for routine clinical workflows. Deep learning has substantially reduced this burden.

However, the fundamental data bottleneck persists because MVSeg2023 provides only 105 annotated training volumes. p. 0
Where the paper says it

Despite this progress, the fundamental data bottleneck of medical image segmentation persists: MVSeg2023 provides only 105 annotated training volumes, an order of magnitude smaller than the datasets that power state-of-the-art segmentation frameworks such as nnU-Net in other anatomical domains [6]. Because 3D TEE annotation requires simultaneous expertise in cardiac anatomy and ultrasound interpretation, this scarcity is unlikely to be resolved through additional manual labelling alone.

The authors address this by using each 2D image purely as a style reference and applying Fourier Domain Adaptation (FDA) to transfer selected low-frequency amplitude components onto a real, already-annotated 3D TEE volume. p. 1
Where the paper says it

We address both obstacles with a single procedure. Rather than treating synthetic images as independent training examples requiring annotation, we use each 2D image purely as a style reference and apply Fourier Domain Adaptation (FDA) [12] to transfer selected low-frequency amplitude components onto a real, already-annotated 3D TEE volume.

This procedure turns the 2D image-to-3D-mask annotation problem into a label-preserving appearance-transfer problem, yielding additional labelled 3D training volumes without new manual segmentation. p. 1
Where the paper says it

Because FDA retains the real-image phase, the volume’s spatial organization is expected to remain largely unchanged, so we reuse the original expert mask rather than claiming that every anatomical feature is exactly preserved. This turns the 2Dimage-to-3D-mask annotation problem into a label-preserving appearance-transfer problem, yielding additional labelled 3D training volumes without new manual segmentation.

How the Training Signal is Built

The methodology involves constructing three generator-guided FDA training sets derived from ChatGPT (GPT-Image), Google Gemini (Nano Banana), and Microsoft MAI-Image-2.6. For each generator, 40 synthetic 2D images of the midesophageal TEE mitral valve view are generated using structured prompts that specify anatomical details and pathology. p. 2

Where the paper says it

Synthetic Image Generation For each of three generators, ChatGPT (GPT-Image), Google Gemini (Nano Banana), and Microsoft MAI-Image2. 6, we generated 40 synthetic 2D images of the midesophageal TEE mitral valve view.

Slice-wise 2D FDA is applied independently to each slice of a real training volume, where the central low-frequency region's amplitude spectrum is replaced with the donor's components while retaining the slice’s phase spectrum. The original expert mask is reused unchanged because the real phase is retained, allowing generated 2D images to add appearance diversity to 3D training without synthetic 3D masks. p. 2
Where the paper says it

Because the real phase is retained, spatial organization is expected to remain largely intact, so Mr was reused unchanged: the donor contributes low-frequency appearance variation while supervision is inherited from the real volume. This lets generated 2D images add appearance diversity to 3D training without synthetic 3D masks or additional annotation.

Where the Gains Come From

The results show that while boundary and surface error metrics were significantly reduced for all three generator-guided conditions relative to the real-only baseline, the Generalized Dice Score (GDS) effect was generator-dependent. Specifically, MAI-FDA had the numerically highest mean GDS (0. p. 3

Where the paper says it

The four FDA-based conditions (ChatGPT- , Gemini-, MAI-, and Non-AI FDA) substantially reduced mean HD95 and Surface Distance relative to the real-only baseline, while their overlap results differed. MAI-FDA had the numerically highest mean GDS (0.

8309 ± 0.0129), while Gemini-FDA showed no significant GDS difference compared to the primary baseline. p. 0
Where the paper says it

ChatGPT-FDA showed a significant GDS advantage over the optimization-matched control but not over the primary baseline, and Gemini-FDA showed no significant GDS difference. Notably, only MAI-FDA provided a statistically robust and significant increase in GDS compared to the primary baseline.

The authors conclude that FDA remains an important augmentation mechanism because the non-AI control performed comparably to ChatGPT-FDA, and generative-AI donors offer a practical means of acquiring style diversity without new annotations

Limitations and Future Directions

The study notes several limitations, including that MVSeg2023 is anonymized, meaning independence across cases is assumed rather than verified. Furthermore, only three runs per condition limits inference on between-run variability. p. 5

Where the paper says it

Second, FDA alters appearance statistics but creates no new valve anatomy or pathology. Third, only three runs per condition limit inference on between-run variability.

The FDA parameter β sensitivity analysis indicated substantial between-run variability and did not support a universal claim that one tested β value is optimal. p. 5
Where the paper says it

In other words, generative-AI donor may be particularly useful when additional real appearance donors are unavailable or costly to collect. Finally, The FDA parameter β sensitivity analysis therefore points to substantial between-run variability and does not support a universal claim that one tested β value is optimal.

External validation on an independent 3D TEE dataset is needed because the test cohort only has 40 real MVSeg2023 volumes. p. 5
Where the paper says it

Also, the test cohort has only 40 real MVSeg2023 volumes. Thus, external validation on an independent 3D TEE dataset is needed.

The findings suggest that generative-AI donor content is not uniquely superior to real-image donors, supporting FDA as an important mechanism for acquiring annotation-free 2D appearance variation Real training-image style donors yielded GDS and HD95 similar to ChatGPT-FDA, so FDA is an important mechanism but generative-AI donor content is not shown to be uniquely beneficial. p. 5
Where the paper says it

Real training-image style donors yielded GDS and HD95 similar to ChatGPT-FDA (no significant difference), so FDA is an important mechanism but generative-AI donor content is not shown to be uniquely beneficial. Its practical value is as an annotation-free source of 2D appearance variation transferable to labelled real volumes, rather than a generator of valid 3D volumes or labels.

The FDA parameter β sensitivity analysis points to substantial between-run variability and does not support a universal claim that one tested β value is optimal. p. 5
Where the paper says it

In other words, generative-AI donor may be particularly useful when additional real appearance donors are unavailable or costly to collect. Finally, The FDA parameter β sensitivity analysis therefore points to substantial between-run variability and does not support a universal claim that one tested β value is optimal.

The GDS conclusions differ between the 390-update comparison and the 540-update real-only control, suggesting overlap and boundary metrics should be interpreted separately

Conclusion

We evaluated general-purpose generative-AI images as 2D appearance donors for label-preserving, slice-wise Fourier Domain Adaptation in 3D mitral-valve segmentation. In the standardized six-condition three-run comparison, MAI-FDA achieved the numerically highest mean GDS (0. p. 5

Where the paper says it

CONCLUSION We evaluated general-purpose generative-AI images as 2D appearance donors for label-preserving, slice-wise Fourier Domain Adaptation in 3D mitral-valve segmentation. In the standardized six-condition three-run comparison, MAI-FDA achieved the numerically highest mean GDS (0.

8309 ± 0.0129), while HD95 and Surface Distance were significantly reduced for all three generator-guided conditions. p. 5
Where the paper says it

00123), whereas HD95 and Surface Distance were significantly reduced for all three generator-guided conditions. The most reproducible effect is thus reduced boundary and surface error, while improved overlap is not shared across generators.

The most reproducible effect is thus reduced boundary and surface error, while improved overlap is not shared across generators. p. 5
Where the paper says it

00123), whereas HD95 and Surface Distance were significantly reduced for all three generator-guided conditions. The most reproducible effect is thus reduced boundary and surface error, while improved overlap is not shared across generators.

Generative models nonetheless offer a practical, annotation-free source of 2D appearance diversity that can be transferred to already labelled real 3D volumes without requiring synthetic voxel-level masks. p. 5
Where the paper says it

Since a non-AI FDA control using real training images as style donors performed comparably on GDS and HD95, this supports FDA as an important augmentation mechanism without establishing that generativeAI donors are superior to real-image donors. Generative models nonetheless offer a practical, annotation-free source of 2D appearance diversity that can be transferred to alreadylabelled real 3D volumes without requiring synthetic voxellevel masks.

Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.

Improvements for AI systems

  1. Fourier Domain Adaptation can be used to transfer low-frequency amplitude components onto a real, already-annotated 3D TEE volume using slice-wise 2D FDA to bridge the dimensional mismatch between 2D generated images and 3D segmentation training. This allows the system to acquire additional labelled 3D training volumes without new manual segmentation.

  2. The system can be configured with six experimental conditions, including real-only training, traditional augmentation, FDA with each generative source, and a non-AI FDA control using real-image style donors. This provides a rigorous framework for comparing the impact of different style donors.

  3. The model can be trained on 145 volumes formed by combining 40 adapted volumes with 105 real training volumes, resulting in additional labelled 3D training volumes without new manual segmentation.

  4. The system can be evaluated using metrics like the Generalized Dice Score (GDS), the 95th-percentile Hausdorff Distance (HD95) for boundary discrepancy, and Surface Distance for mean surface error. These metrics allow for a comprehensive assessment of both overlap and boundary accuracy.

  5. By using generative-AI images as style donors, the system can acquire practical means of acquiring style diversity without new annotations to augment small medical imaging datasets like MVSeg2023.

  6. The system can be designed to compare the performance of different generative AI sources, as demonstrated by showing that only MAI-FDA provided a statistically robust and significant increase in GDS compared to the primary baseline.

Abstract

Accurate segmentation of the mitral valve from three-dimensional transesophageal echocardiography (3D TEE) is severely restricted by the scarcity of publicly available expert- annotated volumetric datasets. To address this critical limitation, we investigated Fourier Domain Adaptation (FDA), utilizing general-purpose generative-AI images (from platforms like Chat- GPT, Google Gemini, and Microsoft MAI) as annotation-free style donors for 3D mitral valve segmentation. We employed a DeepMitral residual U-Net model on the public MVSeg2023 dataset, comparing six conditions over three runs: real-only training, traditional augmentation, FDA with each generative source, and a non-AI FDA control using real-image style donors, plus an optimization-matched real-only control and a sensitivity analysis. Case-level paired testing showed that FDA with these generative sources led to significant reductions in boundary- and surface-related errors across all three donor conditions relative to the primary real-only baseline. However, the impact on the core segmentation metric, Generalized Dice Score (GDS), proved generator-dependent. ChatGPT-FDA showed a significant GDS advantage over the optimization-matched control but not over the primary baseline, and Gemini-FDA showed no significant GDS difference. Notably, only MAI-FDA provided a statistically robust and significant increase in GDS compared to the primary baseline. The sensitivity was not robust across training runs. These findings confirm that FDA remains an important augmentation mechanism. While generative-AI images offer a practical means of acquiring style diversity without new annotations, the non-AI control performed comparably to ChatGPT-FDA, so we do not conclude that this approach is superior to training with real-image donors.

Sources

Related papers