Promise and challenges of heart chamber segmentation from non-contrast CT scans using contrastive unpaired image translation: a feasibility study

arXiv:2606.23879 · eess.IV, cs.AI · Submitted 2026-06-22 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: Electrical Engineering. The day in electrical engineering: signals, systems and the hardware that carries them.

Rosa: I'm Rosa.

Dev: And I'm Dev, here with Taro.

Rosa: We're looking at the paper: "Promise and challenges of heart chamber segmentation from non-contrast CT scans using contrastive unpaired image translation".

Dev: Here's the gist of the paper. Promise and challenges of heart chamber segmentation from non-contrast CT scans using contrastive unpaired image translation: a feasibility study is a feasibility study to evaluate and demonstrate how to segment heart chambers from non-contrast CT scans using contrastive unpaired image translation and deep learning-based segmentation The segmentation model achieved DSC of 0.94 (±0.01), 0.91 (±0.04), 0.92 (±0.03), and 0.93 (±0.02) on synthesized non-contrast images for LA, LV, RA, and RV respectively However, volume errors, particularly for LV and RV, indicate that further refinement and validation are needed before clinical use.

Rosa: Framework Development The framework developed is called ChameleonNet, which utilizes a two-stage approach combining contrastive unpaired image translation with deep learning-based segmentation The first stage involves an unsupervised image translation network trained to generate synthesized noncontrast CT images from contrast-enhanced CT scans while maintaining anatomical details This translation network employs the Contrastive Unpaired Translation (CUT) network with a decoupled contrastive learning (DCL) loss to synthesize noncontrast CT from contrast CT scans The segmentation stage then uses a modified nnU-Net model trained on the synthesized non-contrast images to segment four heart chambers: the left atrium (LA), left ventricle (LV), right atrium (RA), and right ventricle (RV).

Dev: Training Signal Construction The image translation stage utilizes a patch-wise contrastive loss, specifically the decoupled contrastive learning loss (DCE) which is introduced by removing the positive pair term from the denominator of InfoNCE This adjustment leads to the NPC multiplier being equal to 1 when training with lDCE The full loss for the image translation model is formulated as a combination of adversarial loss and the decoupled contrastive learning losses.

Rosa: In the segmentation stage, the modified nnU-Net incorporates a Hausdorff distance loss term alongside cross-entropy loss and Dice loss to minimize prediction errors.

Paper summary: Rosa: To wrap up this paper, "Promise and challenges of heart chamber segmentation from non-contrast CT scans using contrastive unpaired image translation: a feasibility study", the authors are essentially showing that you can use synthesized data to transfer annotations to train a segmentation model on non contrast CT scans >

Dev: The main point they are making is that even though the results show high Dice scores on synthetic images, the volume errors for chambers like the left ventricle and right ventricle signal that we still have work to do before this can be used clinically >

Taro: So, it's a feasible way to estimate cardiac chamber volumes from non contrast CT scans, but we need to address those chamber specific volume errors before you can rely on it for patient diagnosis >

Rosa: That's the title of the paper itself. It sets up both a promise and a challenge, showing that while the technique works on synthetic data, the real world has its own complexities >

Dev: The implications are that this framework offers a reliable method with potential applications in the prognosis and diagnosis of cardiovascular diseases because it lets us estimate volumes where manual annotation is too difficult >

Taro: It opens up possibilities for screening tools that can work with cheaper, more accessible non contrast CT data, which is a big deal since contrast scans aren't always an option >

Rosa: That’s the big picture. It's a reliable path forward in cardiovascular imaging when you can't rely on manual labeling to get the training data you need >

Conclusion: Rosa: So, we've looked at how they tried to map out heart chambers using noncontrast CT scans without having those manual outlines, and this paper is all about that promise versus the real challenges they found >

Dev: Right, it's about taking contrast CT data and turning it into synthetic noncontrast images to train a segmentation model on new data. They called it ChameleonNet >

Taro: The core idea is using an image translation network to synthesize those missing images while keeping all the anatomy intact so the segmentation model gets good training examples >

Rosa: And they achieved decent Dice scores on those synthetic images for things like the left atrium, left ventricle, right atrium, and right ventricle >

Dev: But here's where it gets tricky: they saw volume errors showing up particularly for the left ventricle and the right ventricle >

Taro: That means even though it works well on computer-generated pictures, when they test it on real noncontrast scans, those volume errors mean you can't just trust the numbers yet >

Rosa: So, what does this whole study really suggest about using this approach for actual patient care? >

Dev: It suggests this method is a feasible way to get started with estimating cardiac chamber volumes from noncontrast CT scans without needing every single manual label upfront >

Taro: It's a solid framework that can help in the prognosis and diagnosis of heart conditions because it lets you get those volumes where getting them manually is just too much work >

Rosa: But we have to remember the caveat they pointed out about the real data, right? >

Jing Wang, Tong Yu, Hao-En Lu, Zixue Zeng, Joseph K. Leader, Xin Meng, Jianbing Zhu, Jiantao Pu

Department of Radiology, University of Pittsburgh Department of Bioengineering, University of Pittsburgh Department of Ophthalmology

eess.IV, cs.AI

Submitted: 2026-06-22

Updated: 2026-06-22

DOI: 10.1117/1.JMI.13.5.054004

License: http://creativecommons.org/licenses/by/4.0/

Importance score: 78/100

The gist: Promise and challenges of heart chamber segmentation from non-contrast CT scans using contrastive unpaired image translation…

Key concepts

Contrastive Unpaired Translation (CUT)
This is an unsupervised image translation network designed to create synthetic non-contrast CT images from contrast-enhanced ones without needing paired data. It uses a specific loss function called decoupled contrastive learning to ensure the generated images retain important anatomical details while changing the contrast level.
nnU-Net
This is a modified deep learning model used for segmentation. It is an architecture known for its strong performance in medical image segmentation tasks. In this study, it was trained on the synthesized non-contrast images to accurately delineate the boundaries of the four heart chambers.
Decoupled Contrastive Learning Loss (DCE)
This specific loss function adjusts how the contrastive learning is calculated during image translation. By removing a term from the denominator, it addresses negative-positive coupling, which improves how quickly and effectively the generator learns to produce realistic target images.
Dice Similarity Coefficient (DSC)
The DSC is a metric used to evaluate segmentation performance. It measures how well the predicted boundaries of a heart chamber match the true boundaries in the ground truth data. A higher DSC indicates better agreement between what the model predicted and what is actually there.

Terminology

Summary

Promise and challenges of heart chamber segmentation from non-contrast CT scans using contrastive unpaired image translation: a feasibility study is a feasibility study to evaluate and demonstrate how to segment heart chambers from non-contrast CT scans using contrastive unpaired image translation and deep learning-based segmentation The segmentation model achieved DSC of 0. p. 0

Where the paper says it

1 Promise and challenges of heart chamber segmentation from non-contrast CT scans using contrastive unpaired image translation: a feasibility study Jing Wang1 , Tong Yu1,2, Hao-En Lu1 , Zixue Zeng 1, 2, Joseph K. Leader 1 , Xin Meng1 , Jianbing Zhu1 , Jiantao Pu1, 2, 3 1 Department of Radiology, University of Pittsburgh, Pittsburgh, PA 15213, USA 2 Department of Bioengineering, University of Pittsburgh, Pittsburgh, PA 15213, USA 3 Department of Ophthalmology, University of Pittsburgh, Pittsburgh, PA 15213, USA *Corresponding authors and guarantors of the entire manuscript: Jiantao Pu, PhD Contact Phone: (412) 641-2571 E-mail address: jip13@pitt.

94 (±0.01), 0.91 (±0.04), 0.92 (±0.03), and 0.93 (±0.02) on synthesized non-contrast images for LA, LV, RA, and RV respectively However, volume errors, particularly for LV and RV, indicate that further refinement and validation are needed before clinical use. p. 1
Where the paper says it

Conclusions: ChameleonNet demonstrated feasibility for heart chamber segmentation from noncontrast CT without manual non-contrast annotations. However, volume errors, particularly for LV and RV, indicate that further refinement and validation are needed before clinical use.

Framework Development

The framework developed is called ChameleonNet, which utilizes a two-stage approach combining contrastive unpaired image translation with deep learning-based segmentation The first stage involves an unsupervised image translation network trained to generate synthesized noncontrast CT images from contrast-enhanced CT scans while maintaining anatomical details This translation network employs the Contrastive Unpaired Translation (CUT) network with a decoupled contrastive learning (DCL) loss to synthesize noncontrast CT from contrast CT scans The segmentation stage then uses a modified nnU-Net model trained on the synthesized non-contrast images to segment four heart chambers: the left atrium (LA), left ventricle (LV), right atrium (RA), and right ventricle (RV).

Training Signal Construction

The image translation stage utilizes a patch-wise contrastive loss, specifically the decoupled contrastive learning loss (DCE) which is introduced by removing the positive pair term from the denominator of InfoNCE This adjustment leads to the NPC multiplier being equal to 1 when training with lDCE The full loss for the image translation model is formulated as a combination of adversarial loss and the decoupled contrastive learning losses. In the segmentation stage, the modified nnU-Net incorporates a Hausdorff distance loss term alongside cross-entropy loss and Dice loss to minimize prediction errors. p. 9

Where the paper says it

The standard nnU-Net used the default combined cross-entropy and Dice loss function. The modified nnU-Net additionally incorporated a Hausdorff distance loss term, such that model predictions were compared to ground truth using cross-entropy loss, Dice loss, and Hausdorff distance loss.

Performance Evaluation Metrics

Performance in image translation was compared using Structural Similarity Index Measure (SSIM) and Fréchet inception distance (FID) metrics For segmentation, the Dice similarity coefficient (DSC) measures agreement between predicted and ground truth, while the 95th Hausdorff distance (HD95) measures surface distance. On real non-contrast CT scans, Pearson correlations were reported ranging from 0. p. 1

Where the paper says it

21) mm on synthesized non-contrast images for LA, LV, RA, and RV, respectively. On real non-contrast CT scans, Pearson correlations were 0.

82 to 0.93 for the four chambers. p. 1
Where the paper says it

Approach: We developed ChameleonNet, a framework utilizing the Contrastive Unpaired Translation (CUT) network with decoupled contrastive learning (DCL) loss to synthesize noncontrast CT from contrast CT scans. Using annotations of four heart chambers (left atrium (LA), left ventricle (LV), right atrium (RA), and right ventricle (RV)) from contrast scans, we trained a Hausdorff distance loss-enhanced nnU-Net on synthesized non-contrast images.

Key Findings and Limitations

The results demonstrate feasibility for heart chamber segmentation from noncontrast CT without manual non-contrast annotations However, the study notes that volume errors, particularly for LV and RV, indicate that further refinement and validation are needed before clinical use. The Hausdorff distance loss provided limited improvement on synthetic test images but improved volume agreement when applied to real non-contrast CT scans. p. 1

Where the paper says it

The segmentation model was trained with 292 synthesized non-contrast scans. Performance was evaluated using Dice similarity coefficient (DSC) and 95th Hausdorff distance (HD95) on 36 synthesized non-contrast scans, and volume agreement on 36 real non-contrast CT scans was assessed using Pearson correlation, mean absolute percentage error (MAPE), and mean percentage error (MPE).

A limitation noted is that ECG-gated paired contrast and non-contrast CT scans were not available for testing, which prevents direct Dice-based evaluation on real non-contrast CT scans. p. 4
Where the paper says it

Notably, although the paired contrast and non-contrast CT scans were acquired from the same patients, they were not ECGgated, resulting in misaligned image pairs. Data splitting for model training, evaluation and testing.

Conclusion

The study proposes a novel framework that synthesizes noncontrast CT images from contrast CT scans to transfer annotations and train a segmentation model on synthesized data The validation results on real noncontrast CT scans demonstrate the feasibility of estimating cardiac chamber volumes on non-contrast CT scans. However, chamber-specific volume errors, particularly for the LV and RV, indicate that performance on real non-contrast CT remains imperfect. p. 1

Where the paper says it

Conclusions: ChameleonNet demonstrated feasibility for heart chamber segmentation from noncontrast CT without manual non-contrast annotations. However, volume errors, particularly for LV and RV, indicate that further refinement and validation are needed before clinical use.

The authors conclude that this approach offers a reliable method with potential applications in the prognosis and diagnosis of cardiovascular diseases. p. 3
Where the paper says it

Next, a modified nnU-Net model was trained on the synthesized non-contrast CT scans to segment four heart chambers: the left atrium (LA), left ventricle (LV), right atrium (RA), and right ventricle (RV). By leveraging accurate annotations from contrast-enhanced CT scans and eliminating the need for unreliable labels on non-contrast CT scans, our approach offers a reliable method for estimating cardiac chambers on non-contrast CT scans, with potential applications in the prognosis and diagnosis of cardiovascular diseases.

Pruning redundant reasoning tokens

The image translation stage requires the preservation of anatomical structural integrity, as any distortion could compromise diagnostic reliability and undermine the practical utility of the translated images While similarity metrics (e.g., pixel-wise error) can help maintain structure integrity in paired translation tasks, they present challenges in unpaired translation tasks. p. 5

Where the paper says it

In the image translation stage, the goal is to translate contrast-enhanced images 𝐼௖ into synthetic non-contrast images 𝐼௡௖, while preserving anatomical details. Using unpaired datasets of 𝐼௖ and 𝐼௡௖, the translation network learns the feature distribution of non-contrast images and generates synthetic non-contrast CT images that closely resemble real non-contrast images without compromising the structure integrity of the original contrast-enhanced images.

How the training signal is built

The generator takes a contrast-enhanced image as input and generates a synthetic noncontrast image Feature patches from multiple layers of both the input contrast image and the synthesized non-contrast image are extracted, and a multi-layer patch-wise contrastive loss is computed. The multilayer decoupled contrastive learning loss is then formulated as an expectation over the generated images. p. 8

Where the paper says it

Denoting 𝜈 ௟ ௡ for feature of the 𝑛th location at layer 𝑙, 𝑣௟ ே\௡ for features at other locations, the multilayer features for 𝑥 are represented with {𝑣௟}௅ and the multilayer feature for 𝑦ො is {𝑣ො௟}௅ , where 𝑙 ∈ {1,2, … , 𝐿}. The multilayer decoupled contrastive learning loss is then formulated as: ℒDCE(𝑋) = 𝔼௫~௑෍ ෍ℓDCE(𝑣ො௟ ௡ , 𝑣௟ ௡ , 𝑣௟ ே\௡ ) ே ௡ୀଵ ௅ ௟ୀଵ (5) An identity loss is also incorporated for images in domain 𝑌 to prevent unnecessary alterations introduced by the translation network: ℒDCE(𝑌) = 𝔼௬~௒෍ ෍ℓDCE(𝑣ො௟ ௡ , 𝑣௟ ௡ , 𝑣௟ ே\௡ ) ே ௡ୀଵ ௅ ௟ୀଵ (6) 2.

Where the gains come from

The decoupled contrastive learning loss addresses the negative-positive coupling (NPC) effect observed in InfoNCE loss, leading to reduced learning efficiency This allows the generator to quickly learn to produce realistic images that match the target image distribution. The Hausdorff distance loss improves robustness to the difference in intensity distribution between synthetic and real noncontrast CT. p. 6

Where the paper says it

The discriminator distinguishes between real and synthetic non-contrast images. Stage 2: A modified nnU-Net segments LA, LV, RA, and RV from the synthetic non-contrast image, minimizing Dice, crossentropy, and an additional Hausdorff distance loss between predictions and the contrast image annotations.

This effect is most notable for chambers like LV, where boundary delineation depends on subtle soft tissue contrast. p. 2
Where the paper says it

Moreover, accurate segmentation of heart substructures is valuable in optimizing radiation therapy5 , as dose to heart substructures is linked to adverse outcomes including non-cancer death6 . However, manual delineation of heart chambers on non-contrast CT scans is considerably more challenging than on contrast-enhanced CT scans, due to the poor contrast between adjacent soft tissues, which may increase interobserver variability.

The method also provides superior translation quality and approximately 39% faster training speed compared to CycleGAN. p. 3
Where the paper says it

By leveraging accurate annotations from contrast-enhanced CT scans and eliminating the need for unreliable labels on non-contrast CT scans, our approach offers a reliable method for estimating cardiac chambers on non-contrast CT scans, with potential applications in the prognosis and diagnosis of cardiovascular diseases. The key features of this work are: (1) the use of a decoupled contrastive learning loss within the CUT framework, which accelerates translation training by eliminating the negative-positive coupling effect; (2) a large-scale training dataset with a fully independent test set of real noncontrast CT scans; and (3) a CT-to-CT domain adaptation approach that preserves the native resolution and HU calibration of CT imaging.

This feasibility study has several strengths, including the use of a relatively large dataset and a CT-to-CT translation design that preserves native CT resolution and intensity information Finally, the 2D slicebased translation approach reduces memory requirements and supports efficient training. The study concludes that the modified nnU-Net was selected for subsequent comparisons due to its improved performance on real non-contrast CT scans. p. 5

Where the paper says it

Using unpaired datasets of 𝐼௖ and 𝐼௡௖, the translation network learns the feature distribution of non-contrast images and generates synthetic non-contrast CT images that closely resemble real non-contrast images without compromising the structure integrity of the original contrast-enhanced images. In the image segmentation stage, a modified nnU-Net model was trained to segment LA, LV, RA, and RV, using the synthetic non-contrast image and the annotations of the heart chambers on contrastenhanced images.

Acknowledgement

This work is supported in part by research grants from the National Institutes of Health (NIH) (R01CA237277 and R01HL174570). The code of this study can be accessed through Github repositories. p. 0

Where the paper says it

1 Promise and challenges of heart chamber segmentation from non-contrast CT scans using contrastive unpaired image translation: a feasibility study Jing Wang1 , Tong Yu1,2, Hao-En Lu1 , Zixue Zeng 1, 2, Joseph K. Leader 1 , Xin Meng1 , Jianbing Zhu1 , Jiantao Pu1, 2, 3 1 Department of Radiology, University of Pittsburgh, Pittsburgh, PA 15213, USA 2 Department of Bioengineering, University of Pittsburgh, Pittsburgh, PA 15213, USA 3 Department of Ophthalmology, University of Pittsburgh, Pittsburgh, PA 15213, USA *Corresponding authors and guarantors of the entire manuscript: Jiantao Pu, PhD Contact Phone: (412) 641-2571 E-mail address: jip13@pitt.

The data cannot be shared publicly as they consist of identifiable patient information obtained from institutional hospital databases. p. 7
Where the paper says it

Other patches in 𝑥 are treated as negative samples with feature vector 𝜈 ି. The loss is designed to maximize the mutual information between the anchor and its positive sample while maximizing the distance to the negative samples, facilitating learning the shared features between the source and target domains.

References

The references listed support the claims made throughout the paper. The references include works on image translation methods like CycleGAN and CUT, as well as deep learning segmentation models like nnU-Net. p. 9

Where the paper says it

CUT and CycleGAN were trained with default settings16,19, except for the 1024 patch sampling in CUT. For segmentation, the nnU-Net model was trained with random 90*160*60 image crops, using pixel spacing [2.

The references also cover the use of Hausdorff distance loss in medical image segmentation and the challenges associated with unpaired image translation tasks. p. 1
Where the paper says it

2 ABSTRACT Purpose: To evaluate the feasibility and challenges of heart chamber segmentation from noncontrast CT scans using contrastive unpaired image translation and deep learning-based segmentation. Approach: We developed ChameleonNet, a framework utilizing the Contrastive Unpaired Translation (CUT) network with decoupled contrastive learning (DCL) loss to synthesize noncontrast CT from contrast CT scans.

The references also include studies on cardiac volume variation throughout the cardiac cycle.

Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.

Improvements for AI systems

  1. Decoupled contrastive learning loss within ChameleonNet accelerates translation training by eliminating the negative-positive coupling effect, which improves efficiency compared to standard InfoNCE loss in CUT. This allows the generator to quickly learn to produce realistic images that match the target image distribution.

  2. Modified nnU-Net incorporating a Hausdorff distance loss term enhances segmentation robustness, as it showed meaningful improvements on real non-contrast CT scans and improved volume agreement for selected chambers, particularly the LV. This regularization helps in boundary delineation where subtle soft tissue contrast is limited.

  3. The ChameleonNet framework enables heart chamber segmentation from non-contrast CT scans without manual annotations by using synthesized images to train a deep learning model. This approach allows the system to transfer annotations from contrast CT to non-contrast CT without the need for paired data.

Abstract

Purpose: To evaluate the feasibility and challenges of heart chamber segmentation from non-contrast CT scans using contrastive unpaired image translation and deep learning-based segmentation. Approach: We developed ChameleonNet, a framework utilizing the Contrastive Unpaired Translation (CUT) network with decoupled contrastive learning (DCL) loss to synthesize non-contrast CT from contrast CT scans. Using annotations of four heart chambers (left atrium (LA), left ventricle (LV), right atrium (RA), and right ventricle (RV)) from contrast scans, we trained a Hausdorff distance loss-enhanced nnU-Net on synthesized non-contrast images. The translation model was trained with 35,538 contrast-enhanced and 37,197 non-contrast CT slices. The segmentation model was trained with 292 synthesized non-contrast scans. Performance was evaluated using Dice similarity coefficient (DSC) and 95th Hausdorff distance (HD95) on 36 synthesized non-contrast scans, and volume agreement on 36 real non-contrast CT scans was assessed using Pearson correlation, mean absolute percentage error (MAPE), and mean percentage error (MPE). Results: The segmentation model achieved DSC of 0.94 (0.01), 0.91 (0.04), 0.92 (0.03), 0.93 (0.02), and HD95 of 3.63 (1.49), 5.74 (4.08), 5.18 (1.77), 5.51 (3.21) mm on synthesized non-contrast images for LA, LV, RA, and RV, respectively. On real non-contrast CT scans, Pearson correlations were 0.93, 0.82, 0.87, and 0.89 (all p<0.001), with MAPE ranging from 9.22% to 20.79%, and MPE ranging from-12.52% to 4.67%. Conclusions: ChameleonNet demonstrated feasibility for heart chamber segmentation from non-contrast CT without manual non-contrast annotations. However, volume errors, particularly for LV and RV, indicate that further refinement and validation are needed before clinical use.

Related papers