Uncertainty Propagation for Echocardiography Clinical Metric Estimation via Contour Sampling

The gist: This work proposes a novel uncertainty estimation method based on contouring rather than segmentation to propagate uncertainty from input images to derived clinical metrics in echocardiography.

Contour-based Aleatoric Uncertainty Modeling

Contrary to most previous work in uncertainty estimation for segmentation, this approach models the uncertainty of contour points rather than the pixel-wise uncertainty, advocating that this is more appropriate for left-ventricle segmentation because there is no uncertainty around the presence of the left-ventricle in the image, but rather uncertainty regarding its position<ref:2502.12713#pg4>. The prediction of a contour and its associated uncertainty begins with a training dataset made of D = N pairs n=1, where each pair is an image xn ∈ R H×W of height H and width W, and corresponding ground truth segmentation map yn ∈ 0, 1 C×H×W <ref:2502.12713#pg4>. A contouring neural network f c θ (·) is trained to predict K heatmaps Z k ∈ R H×W, each associated to a control point, as illustrated on the left of Figure 1 <ref:2502.12713#pg5>. Each heatmap contains a blurry white spot whose center corresponds to the position of a control point (variable µ) and whose halo size corresponds to the uncertainty of that point (variable Σ) <ref:2502.12713#pg5>. The mean and variance can be extracted from the heatmap using two coordinate maps I ∈ R H×W and J ∈ R H×W, which are used to extract the mean and variance of the heatmap for each point <ref:2502.12713#pg5>. By combining the computed variance and covariance values, a 2-dimensional Gaussian distribution can be modeled<ref:2502.12713#pg5>. The model thus predicts a tuple (ˆµ, Σ) ˆ for each image x, with µˆ ∈ R K×2 and Σˆ ∈ R K×2×2 <ref:2502.12713#pg5>. To train the network, the negative-log likelihood of the Gaussian is used for each point<ref:2502.12713#pg5>.

Shape Uncertainty Modeling

While the previous formulation allows predicting one contour, uncertainty propagation to clinical metrics requires multiple contours, which can be retrieved with stochastic sampling of ˆµk, Σˆk K k=1 <ref:2502.12713#pg6>. However, sampling each individual distribution does not provide valid contours as the inter-point covariance is not modeled<ref:2502.12713#pg6>. We propose using a statistical shape model that models the uncertainty of points given the position of other points, defined as a posterior shape model (PSM) based on principal component analysis (PCA)<ref:2502.12713#pg6>. The PSM predicts a conditional distribution given partial data, p(ss(g)) = N (µc, Σc) <ref:2502.12713#pg7>. The mean and covariance matrix are given by µc = ¯µ + Q(Q T g Qg + ϵ squared I2K) −1Q T g(s(g) − µg) ∈ R 2K, (9)<ref:2502.12713#pg7>. The model will refer to the matrix Q computed with respect to µˆ as Qˆ <ref:2502.12713#pg7>.

Aleatoric and Shape Uncertainty Fusion

A hierarchical sampling algorithm is defined to fuse both the contour uncertainty and shape uncertainty<ref:2502.12713#pg7>. The process begins by sampling both basal points and apex point independently, as these points are tied to well-defined anatomical landmarks<ref:2502.12713#pg7>. Given this partial input, the parameters of the posterior shape model distribution can be predicted using PSM<ref:2502.12713#pg7>. To obtain a distribution conditioned on both the original model prediction and the sampled points, we fuse both distributions by multiplying them as is done when combining a likelihood and prior term<ref:2502.12713#pg7>. This results in new conditioned distributions for each point, which are then used to sample the midpoints between previously sampled points and repeat the process until all points are sampled at each hierarchical level<ref:2502.12713#pg7>.

Uncertainty Propagation

The predictive uncertainty of a clinical metric F is given by Vp(mx) = Eqw(θ) h Vpθ(yx) [F(y)]i + Vqw(θ) h Epθ(yx) [F(y)]i <ref:2502.12713#pg7>. To compute this uncertainty, a Monte Carlo framework is used where Yˆ = Ta to be the number of samples to estimate the aleatoric uncertainty and Te to be the number of epistemic samples <ref:2502.12713#pg7>. For each prediction in Yˆ, a given clinical metric F can be computed to obtain M = m1 1 Ta, m1 2 Ta, …, mTe Te <ref:2502.12713#pg7>. The mean and variance of each subset are computed to obtain M′ = µm1, σ2 m1, … (µmTe, σ2 mTe) <ref:2502.12713#pg7>. Finally, both components of the clinical metric uncertainty can be combined to obtain the predictive uncertainty, σ squared F = σ squared Fa + σ squared Fe <ref:2502.12713#pg7>.

Conclusion

The framework provides an intelligible uncertainty estimation that can combine both aleatoric and epistemic uncertainty<ref:2502.12713#pg10>. Our findings highlight that a contour-based approach to aleatoric uncertainty, combined with shape uncertainty modeling, delivers optimal results in clinical metric calibration<ref:2502.12713#pg10>. This approach is further improved by integrating temporal consistency in the sampling process<ref:2502.12713#pg10>. The incompatibility between the variability required to achieve clinical metric calibration and pixel-wise uncertainty calibration is a limitation of our work<ref:2502.12713#pg10>. CASUS and SSN exhibit better calibration on clinical metrics but show under-confidence in pixel-wise calibration<ref:2502.12713#pg10>.

Clinical Metrics

For each segmentation output, four clinical metrics are computed: the left ventricular area (Area), the leftventricular fractional area change (FAC), the left ventricular volume using the 2CH and 4CH views (Volume), and the ejection fraction (EF)<ref:2502.12713#pg9>. To assess uncertainty estimates, we use the uncertainty calibration error (UCE) [37], defined by UCE = X M m=1 Bm Ntest err(Bm) − uncert(Bm), (23)<ref:2502.12713#pg9>. We exclude predictions with negative values and discard Monte Carlo samples with negative values to ensure that clinically irrelevant samples do not bias the uncertainty calibration error calculation<ref:2502.12713#pg9>.

Experimental Setup

The experimental setup involved comparing various aleatoric uncertainty methods, including pixel-wise aleatoric uncertainty (PWA) [19], stochastic segmentation networks (SSN) [29], and test-time augmentation (TTA) [30]<ref:2502.12713#pg9>. The CASUS+t method combined with MC-Dropout ranks as the top or second-best method across all metrics in both datasets, emphasizing the benefits of temporal consistent sampling and the inclusion of epistemic uncertainty<ref:2502.12713#pg10>. The performance comparison showed that CASUS yields higher mutual information values on the Private dataset, suggesting it captures larger errors more effectively<ref:2502.12713#pg9>.

Evaluation Metrics

To assess segmentation quality, the Dice coefficient is used<ref:2502.12713#pg9>. The metric used to assess the image-level uncertainty is the negative correlation (Corr.) between image-level uncertainty and the image Dice<ref:2502.12713#pg9>. The expected calibration error (ECE) is computed as follows [36], where ECE = X M m=1 Bm Ntest acc(Bm) − conf(Bm) (22)<ref:2502.12713#pg9>.

1. The improved system can estimate uncertainty propagation from input images to segmentation and, subsequently, to the derived clinical metrics by explicitly predicting aleatoric uncertainty directly for contour points rather than relying on pixel-wise uncertainty maps which often performs poorly since independent pixel distributions make plausible segmentation samples unlikely.

2. The system can generate a robust endto-end pipeline for automatic clinical metric estimation with intelligible uncertainty, specifically by implementing the CASUS method, which combines contour aleatoric shape uncertainty sampling and temporal consistent sampling to produce plausible contours.

3. The improved system can provide clinically relevant metrics by employing a Monte Carlo framework where it computes the mean and variance of each subset of clinical metric predictions, resulting in a total uncertainty estimate where the spatial uncertainty can be estimated per pixel by computing entropy of all samples in Yˆ.

Echocardiography plays a fundamental role in the extraction of clinical parameters (e.g. left ventricular volume and ejection fraction) required to determine the presence and severity of heart-related conditions. When deploying automated techniques for computing these parameters, uncertainty estimation is highly valuable for assessing their utility. Since clinical parameters are usually derived from segmentation maps, there is no clear path for converting pixel-wise uncertainty values into uncertainty estimates in the downstream clinical metric calculation. In this work, we propose a novel uncertainty estimation method based on contouring rather than segmentation. Our method explicitly predicts contour location uncertainty from which contour samples can be drawn. Finally, the sampled contours can be used to propagate uncertainty to clinical metrics. Our proposed method not only provides accurate uncertainty estimations for the task of contouring but also for the downstream clinical metrics on one public and two private cardiac ultrasound datasets containing a total of 1264 patients. Code is available at: https://github.com/ThierryJudge/contouring-uncertainty.