Multimodal Deep Learning for Uncertainty-Aware Radiation Pneumonitis Risk Prediction

arXiv:2608.28937 · eess.IV · Submitted 2026-08-28 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: Electrical Engineering. The day in electrical engineering: signals, systems and the hardware that carries them.

Rosa: We're looking at the paper: "Multimodal Deep Learning for Uncertainty-Aware Radiation Pneumonitis Risk Prediction".

Dev: Here's the gist of the paper. Radiation pneumonitis (RP) is a common and clinically significant toxicity of thoracic radiation therapy that can cause pulmonary morbidity and impair quality of life.

Rosa: MM-DiT provides individualized RP risk estimates together with complementary measures of aleatoric, epistemic, and label uncertainty.

Dev: Problem Statement Conventional dose-volume histogram–based metrics and normal tissue complication probability models are widely used for RP risk assessment but inadequately capture the complex spatial, anatomical, and patient-specific factors underlying radiation-induced lung injury.

Rosa: These traditional approaches compress complex three-dimensional (3D) dose distributions into one-dimensional summary statistics, thereby discarding spatial information regarding regional dose deposition and tissue heterogeneity.

Paper summary: Rosa: So, to get into what this paper is really saying, they start by pointing out that conventional dose-volume histogram based metrics and normal tissue complication probability models just can't handle the complexity of radiation lung injury >

Dev: They argue these traditional models struggle because they compress everything into one dimension when there's so much spatial information about the dose distribution that gets lost >

Rosa: The thesis here is that by using a Multimodal Bayesian Diffusion Transformer, they can jointly estimate RP risk and characterize the sources of predictive uncertainty >

Taro: Why does characterizing uncertainty matter for us? Because right now, most AI methods just give you a single number—a point estimate—without telling you how much to trust it >

Dev: The paper claims that this approach offers a more reliable and transparent way to do AI assisted RP risk assessment by providing that explicit modeling of label noise >

Rosa: They’re trying to support more informed individualized treatment planning and safer clinical decision making in thoracic radiation therapy using this framework >

Taro: It's about giving clinicians a tool where they can stratify patients into risk uncertainty groups, like high risk plus high confidence, which guides immediate intervention decisions >

Dev: The paper is essentially showing how integrating heterogeneous sources of information—patient demographics, clinical characteristics, dose distributions, and radiomic features from imaging—can capture the complex interactions contributing to RP development >

Rosa: It’s about moving away from just a deterministic prediction to something that acknowledges the reliability of every individual prediction >

Taro: That's what makes it relevant because in real-world clinical settings, you can't afford to make a decision without knowing the risk and how certain you are about that risk >

Dev: So, this framework is designed to be more reliable and transparent for those who need to use AI in their treatment planning process >

Rosa: The authors of this paper are Jin Yang, Tian Liu, Jing Wang, Robert Samstein, Kenneth Rosenzweig, Julie Bloom and Ming Chao at the Department of Radiation Oncology at Icahn School of Medicine at Mount Sinai in New York City >

Taro: It’s interesting that they’re focusing on lung cancer and other thoracic malignancies as the primary context for this work because RP is such a significant constraint on safe dose delivery >

Dev: The paper also notes that the clinical translation of AI methods remains limited by challenges like generalizability, calibration, interpretation, and reliable assessment of individual patient risk >

Rosa: So, even with improved performance numbers like AUC values reaching eighty-five point five nine percent in the RTOG six hundred seventeen cohort and eighty-nine point one one percent in the MSHS cohort >

Taro: Those performance numbers are good, but they're just one piece of a puzzle when you consider how robust the framework handles those uncertainties when the world misbehaves >

Dev: It shows that integrating multimodal information can improve predictive performance over models that only look at one modality, like just dose distribution or just imaging features >

Rosa: So, this paper is essentially proposing a way to get better predictions by making sure we also understand the uncertainty around those predictions >

Conclusion: Taro: To wrap this up, the Multimodal Deep Learning for Uncertainty-Aware Radiation Pneumonitis Risk Prediction framework is a Bayesian multimodal approach that integrates planning CT imaging and three dee radiation dose distributions for individualized RP risk prediction and explicit uncertainty quantification > <ref:2608.28937#pg2>

Dev: It's about providing a more transparent and reliable approach to AI assisted RP risk assessment by giving us that explicit modeling of label noise >

Rosa: It moves us toward more informed individualized treatment planning because it allows clinicians to see not just the predicted risk but also how certain the model is in its own assessment >

Taro: This framework extends conventional risk stratification toward uncertainty-aware clinical interpretation, allowing you to categorize patients into groups like high risk plus high confidence for immediate intervention considerations >

Dev: It’s a tool that helps identify predictions that are sufficiently supported to inform clinical consideration without necessarily prescribing specific treatment modifications based solely on the model output >

Taro: It’s a framework that gives us a mechanism for identifying predictions that may be sufficiently supported to inform clinical consideration >

Dev: Ultimately, it's about providing a more transparent and reliable approach to AI assisted RP risk assessment >

Jin Yang, Tian Liu, Jing Wang, Robert Samstein, Kenneth Rosenzweig, Julie Bloom

Department of Radiation Oncology, Icahn School of Medicine at Mount Sinai

eess.IV

Submitted: 2026-08-28

Updated: 2026-10-31

Comments: 28 pages, 9 figures, 5 tables

License: http://creativecommons.org/licenses/by/4.0/

Importance score: 92/100

The gist: Radiation pneumonitis (RP) is a common and clinically significant toxicity of thoracic radiation therapy that can cause pulmonary morbidity and impair quality of life.

Key concepts

Multimodal Bayesian Diffusion Transformer (MM-DiT)
This is the core framework that integrates planning CT images and 3D dose distributions. It uses a diffusion transformer structure to learn complex, cross-modal representations from these different data types without needing prior toxicity labels. This allows the model to understand how anatomy and dose interact.
Aleatoric Uncertainty (Ualea)
This uncertainty reflects the inherent randomness or stochasticity present in the input data itself. It captures how noisy or variable the patient's characteristics are, which directly impacts the prediction's variance. High aleatoric uncertainty means the model is inherently less certain about that specific prediction.
Epistemic Uncertainty (Uepst)
This uncertainty measures limitations in the model itself, stemming from insufficient or unrepresentative training data. It indicates that a patient's characteristics fall outside what the model has seen before. High epistemic uncertainty suggests the model is unsure because it hasn't encountered similar cases.
Label Uncertainty (Ulabel)
This component explicitly models inconsistencies and subjectivity in the toxicity annotations used for training. It quantifies how much the true label might fluctuate based on the predicted risk, providing a measure of confidence in the ground truth data.

Terminology

Summary

Radiation pneumonitis (RP) is a common and clinically significant toxicity of thoracic radiation therapy that can cause pulmonary morbidity and impair quality of life. p. 0

Where the paper says it

chao@mountsinai. org; Abstract Radiation pneumonitis (RP) is a common and clinically significant toxicity of thoracic radiation therapy that can cause pulmonary morbidity and impair quality of life.

MM-DiT provides individualized RP risk estimates together with complementary measures of aleatoric, epistemic, and label uncertainty. p. 0
Where the paper says it

A learnable label-noise model is incorporated to explicitly account for uncertainty arising from imperfect toxicity annotations. MM-DiT, therefore, provides individualized RP risk estimates together with complementary measures of aleatoric, epistemic, and label uncertainty, enabling assessment of prediction reliability at the individual-patient level.

Problem Statement

Conventional dose-volume histogram–based metrics and normal tissue complication probability models are widely used for RP risk assessment but inadequately capture the complex spatial, anatomical, and patient-specific factors underlying radiation-induced lung injury. These traditional approaches compress complex three-dimensional (3D) dose distributions into one-dimensional summary statistics, thereby discarding spatial information regarding regional dose deposition and tissue heterogeneity. p. 1

Where the paper says it

While clinically established, these approaches have important limitations. First, DVHs compress complex threedimensional (3D) dose distributions into one-dimensional summary statistics, thereby discarding spatial information regarding regional dose deposition and tissue heterogeneity.

Furthermore, RP is a multifactorial process influenced not only by radiation dose but also by patient-specific characteristics, pulmonary function, tumor location, anatomical imaging features, and individual radiosensitivity. p. 1
Where the paper says it

First, DVHs compress complex threedimensional (3D) dose distributions into one-dimensional summary statistics, thereby discarding spatial information regarding regional dose deposition and tissue heterogeneity. Furthermore, RP is a multifactorial process influenced not only by radiation dose but also by patient-specific characteristics, pulmonary function, tumor location, anatomical imaging features, and individual radiosensitivity [11, 12].

A fundamental limitation of current RP prediction models is that they predominantly use deterministic supervised learning and provide a point estimate of RP risk without quantifying the uncertainty associated with individual predictions. p. 2
Where the paper says it

Although many studies have demonstrated improved predictive performance, their clinical translation remains limited by challenges including generalizability, calibration, interpretation, and reliable assessment of individual patient risk. A fundamental limitation of current RP prediction models is that they predominantly use deterministic supervised learning and provide a point estimate of RP risk without quantifying the uncertainty associated with individual predictions.

Framework Architecture

The proposed Multimodal Bayesian Diffusion Transformer (MM-DiT) framework integrates planning computed tomography (CT) images and three-dimensional radiation dose distributions through self-supervised multimodal pre-training. This framework jointly estimates RP risk and characterizes the sources of predictive uncertainty. p. 0

Where the paper says it

Recent machine learning approaches have improved RP risk prediction by integrating multimodal clinical and imaging information; however, most provide a point risk estimate without quantifying the reliability of individual predictions, limiting their potential clinical utility. We propose a Multimodal Bayesian Diffusion Transformer (MM-DiT) framework that jointly estimates RP risk and characterizes the sources of predictive uncertainty.

The process involves two sequential stages: self-supervised multimodal representation learning and supervised Bayesian fine-tuning. p. 5
Where the paper says it

Patients from two cohorts are independently and randomly divided into training and internal test sets at a ratio of 80% : 20%. MM-DiT is a multimodal deep learning framework that performs RP prediction through two sequential stages: self-supervised multimodal representation learning and supervised Bayesian fine-tuning.

During self-supervised pre-training, a multi-modal Diffusion Transformer (DiT) learns cross-modal representations from planning CT images and 3D dose distributions without using RP labels. p. 5
Where the paper says it

MM-DiT is a multimodal deep learning framework that performs RP prediction through two sequential stages: self-supervised multimodal representation learning and supervised Bayesian fine-tuning. During self-supervised pre-training, a multi-modal Diffusion Transformer (DiT) learns cross-modal representations from planning CT images and 3D dose distributions without using RP labels.

The resulting representations are subsequently refined using a latent diffusion transformer and transferred to a Bayesian prediction framework for probabilistic RP risk estimation. p. 0
Where the paper says it

MM-DiT integrates planning computed tomography (CT) images and three-dimensional radiation dose distributions through self-supervised multimodal pre-training, reducing reliance on limited and potentially noisy toxicity labels. The resulting representations are further refined using a latent diffusion transformer and transferred to a Bayesian prediction framework for probabilistic RP risk estimation.

Uncertainty Quantification

MM-DiT provides complementary measures of aleatoric, epistemic, and label uncertainty. The total predictive uncertainty is decomposed into three distinct components: aleatoric (Ualea), epistemic (Uepst), and label uncertainty (Ulabel). p. 6

Where the paper says it

During inference, stochastic forward passes generate a distribution of RP risk predictions rather than a single point estimate, with the mean of the predictive distribution used as the final RP risk estimate. Predictive uncertainty is decomposed into three components: aleatoric, epistemic, and label uncertainty.

Aleatoric uncertainty captures inherent stochasticity of the input data and its impact on model predictions by averaging the variances of the Bernoulli-distributed probabilities across T stochastic MC Dropout passes. p. 6
Where the paper says it

During inference, stochastic forward passes generate a distribution of RP risk predictions rather than a single point estimate, with the mean of the predictive distribution used as the final RP risk estimate. Predictive uncertainty is decomposed into three components: aleatoric, epistemic, and label uncertainty.

Epistemic uncertainty reflects limitations in the model’s parameters due to insufficient or non-representative training data, typically indicating that a patient possesses characteristics outside the distribution of the training cohort. p. 3
Where the paper says it

Detailed demographic and clinical characteristics for these two cohorts are summarized in Table 1. 2 Data Preprocessing The planning CT images and 3D dose distributions undergo a multi-step preprocessing pipeline prior to model training.

Label uncertainty explicitly models inconsistencies and subjectivity in toxicity annotations by quantifying the expected volatility of the label given the predicted risk and estimated noise rates. p. 0
Where the paper says it

The resulting representations are further refined using a latent diffusion transformer and transferred to a Bayesian prediction framework for probabilistic RP risk estimation. A learnable label-noise model is incorporated to explicitly account for uncertainty arising from imperfect toxicity annotations.

Performance and Utility

Across both independent cohorts, CNN+PoE achieved the highest overall performance, with AUC values reaching 85.59% in the RTOG 0617 cohort and 89. p. 3

Where the paper says it

Each patient record includes a planning CT image set and corresponding 3D spatial dose distribution. The first cohort is derived from the NRG Radiation Therapy Oncology Group (RTOG) 0617 randomized clinical trial and accessed through The Cancer Imaging Archive [34] with approval from the National Cancer Institute (NCI) National Clinical Trials Network (NCTN)/National Community Oncology Research Program (NCORP).

p. 3
Where the paper says it

Each patient record includes a planning CT image set and corresponding 3D spatial dose distribution. The first cohort is derived from the NRG Radiation Therapy Oncology Group (RTOG) 0617 randomized clinical trial and accessed through The Cancer Imaging Archive [34] with approval from the National Cancer Institute (NCI) National Clinical Trials Network (NCTN)/National Community Oncology Research Program (NCORP).

11% in the MSHS cohort. p. 3
Where the paper says it

A total of 416 subjects are therefore included from the RTOG 0617 cohort. The second cohort comprised 213 patients with lung cancer who received thoracic RT at the Mount Sinai Health System (MSHS).

p. 3
Where the paper says it

A total of 416 subjects are therefore included from the RTOG 0617 cohort. The second cohort comprised 213 patients with lung cancer who received thoracic RT at the Mount Sinai Health System (MSHS).

The model calibration analysis showed consistently low calibration error across both cohorts, with an ECE of 0. p. 3
Where the paper says it

We evaluate MM-DiT in two independent cohorts using complementary assessments of predictive discrimination, calibration, and uncertainty quantification. Clinical utility is further evaluated using decision curve analysis to determine whether incorporating uncertainty-aware RP risk estimates could provide greater net benefit across clinically relevant decision thresholds.

0330 in RTOG 0617 and 0. p. 3
Where the paper says it

Each patient record includes a planning CT image set and corresponding 3D spatial dose distribution. The first cohort is derived from the NRG Radiation Therapy Oncology Group (RTOG) 0617 randomized clinical trial and accessed through The Cancer Imaging Archive [34] with approval from the National Cancer Institute (NCI) National Clinical Trials Network (NCTN)/National Community Oncology Research Program (NCORP).

0198 in MSHS. p. 3
Where the paper says it

A total of 416 subjects are therefore included from the RTOG 0617 cohort. The second cohort comprised 213 patients with lung cancer who received thoracic RT at the Mount Sinai Health System (MSHS).

Decision curve analysis demonstrated that MM-DiT achieved a higher net benefit than both the treat-all and treat-none strategies across a broad range of threshold probabilities. p. 3
Where the paper says it

We evaluate MM-DiT in two independent cohorts using complementary assessments of predictive discrimination, calibration, and uncertainty quantification. Clinical utility is further evaluated using decision curve analysis to determine whether incorporating uncertainty-aware RP risk estimates could provide greater net benefit across clinically relevant decision thresholds.

Clinical Interpretation

The framework allows for stratifying patients into four risk–uncertainty groups based on predicted RP probability and overall uncertainty. This stratification includes categories such as High Risk + High Confidence, which suggests immediate clinical intervention. p. 1

Where the paper says it

Preventing RP while maintaining optimal treatment outcomes therefore remains a central challenge in thoracic RT. Accurate identification of patients at high risk for RP before treatment could facilitate early risk stratification and more informed clinical management, providing a foundation for individualized treatment decisions and improved patient outcomes.

The framework is not intended to prescribe specific treatment modifications based solely on model output but provides a mechanism for identifying predictions that may be sufficiently supported to inform clinical consideration. p. 2
Where the paper says it

Consequently, even when a model demonstrates good overall discrimination, its output for a specific patient may not indicate how strongly that prediction is supported by the available evidence. This limitation is particularly important when RP risk estimates are used to inform treatment planning or clinical management, where the reliability of an individual prediction may be as important as its estimated probability [24–29].

Limitations

Several limitations should be considered, including the reliance on pretreatment CT and radiation dose distributions as primary model inputs. The predominance of aleatoric uncertainty observed in the analysis suggests the need for future studies incorporating toxicity-rich longitudinal data acquired during and after RT. p. 3

Where the paper says it

Clinical utility is further evaluated using decision curve analysis to determine whether incorporating uncertainty-aware RP risk estimates could provide greater net benefit across clinically relevant decision thresholds. By combining individualized RP risk prediction with explicit characterization of aleatoric, epistemic, and label uncertainty, MM-DiT provides not only an estimate of RP risk but also information regarding the reliability of that estimate.

p. 3
Where the paper says it

Clinical utility is further evaluated using decision curve analysis to determine whether incorporating uncertainty-aware RP risk estimates could provide greater net benefit across clinically relevant decision thresholds. By combining individualized RP risk prediction with explicit characterization of aleatoric, epistemic, and label uncertainty, MM-DiT provides not only an estimate of RP risk but also information regarding the reliability of that estimate.

Additionally, additional multi-institutional validation across diverse patient populations and imaging protocols is needed to establish generalizability. The risk and uncertainty thresholds used for the four-group stratification require clinical validation before they can be translated into operational decision thresholds. p. 3
Where the paper says it

We evaluate MM-DiT in two independent cohorts using complementary assessments of predictive discrimination, calibration, and uncertainty quantification. Clinical utility is further evaluated using decision curve analysis to determine whether incorporating uncertainty-aware RP risk estimates could provide greater net benefit across clinically relevant decision thresholds.

Conclusion

In this study, MM-DiT developed a Bayesian multimodal framework that integrates planning CT imaging and 3D radiation dose distributions for individualized RP risk prediction and explicit uncertainty quantification. The framework extends conventional RP risk stratification toward uncertainty-aware clinical interpretation. p. 3

Where the paper says it

By combining individualized RP risk prediction with explicit characterization of aleatoric, epistemic, and label uncertainty, MM-DiT provides not only an estimate of RP risk but also information regarding the reliability of that estimate. This uncertainty-aware framework may provide a more informative basis for risk stratification and clinical decision-making, supporting the potential translation of AI-assisted RP prediction into routine thoracic radiation oncology.

Acknowledgment

This manuscript was prepared using data from Datasets from the NCTNNCORP Data Archive of the National Cancer Institute’s (NCI’s) National Clinical Trials Network (NCTN). This work was partially supported by the Radiation Oncology Institute (Grant 2022-9132). p. 3

Where the paper says it

Each patient record includes a planning CT image set and corresponding 3D spatial dose distribution. The first cohort is derived from the NRG Radiation Therapy Oncology Group (RTOG) 0617 randomized clinical trial and accessed through The Cancer Imaging Archive [34] with approval from the National Cancer Institute (NCI) National Clinical Trials Network (NCTN)/National Community Oncology Research Program (NCORP).

References

[1] Rahi, M. S. et al. Radiation-induced lung injury—current perspectives and management. p. 1

Where the paper says it

Accurate identification of patients at high risk for RP before treatment could facilitate early risk stratification and more informed clinical management, providing a foundation for individualized treatment decisions and improved patient outcomes. Current clinical practice for reducing RP risk relies on lung dose-volume histograms (DVHs), which summarize the relationship between radiation dose and the volume of healthy lung receiving that dose.

[2] Kaesmann, L. et al. Radiation-induced lung toxicity–cellular and molecular mechanisms of pathogenesis, management, and literature review. p. 0

Where the paper says it

org; Abstract Radiation pneumonitis (RP) is a common and clinically significant toxicity of thoracic radiation therapy that can cause pulmonary morbidity and impair quality of life. Although conventional dose-volume histogram–based metrics and normal tissue complication probability models are widely used for RP risk assessment, they inadequately capture the complex spatial, anatomical, and patient-specific factors underlying radiation-induced lung injury.

[3] Kraus, K. M. et al. Dosiomics and radiomics-based prediction of pneumonitis after radiotherapy and immune checkpoint inhibition: The relevance of fractionation. p. 0

Where the paper says it

org; Abstract Radiation pneumonitis (RP) is a common and clinically significant toxicity of thoracic radiation therapy that can cause pulmonary morbidity and impair quality of life. Although conventional dose-volume histogram–based metrics and normal tissue complication probability models are widely used for RP risk assessment, they inadequately capture the complex spatial, anatomical, and patient-specific factors underlying radiation-induced lung injury.

[4] Weisman, M. et al. Evaluation of radiation pneumonitis in a phase 2 study of consolidation immunotherapy with nivolumab and ipilimumab or nivolumab alone following concurrent chemoradiation therapy for unresectable stage iiia/iiib nonsmall cell lung cancer. p. 3

Where the paper says it

The first cohort is derived from the NRG Radiation Therapy Oncology Group (RTOG) 0617 randomized clinical trial and accessed through The Cancer Imaging Archive [34] with approval from the National Cancer Institute (NCI) National Clinical Trials Network (NCTN)/National Community Oncology Research Program (NCORP). The trial includes 544 patients with stage III non-small-cell lung cancer (NSCLC) who were enrolled in a randomized study evaluating radiation dose escalation in combination of chemotherapy, with and without cetuximab [35, 36].

[5] Xiao, L. et al. Early prediction of radiation pneumonitis in patients with lung cancer treated with immunotherapy through monitoring of plasma chemokines. p. 1

Where the paper says it

Accurate identification of patients at high risk for RP before treatment could facilitate early risk stratification and more informed clinical management, providing a foundation for individualized treatment decisions and improved patient outcomes. Current clinical practice for reducing RP risk relies on lung dose-volume histograms (DVHs), which summarize the relationship between radiation dose and the volume of healthy lung receiving that dose.

[6] Lyman, J. p. 1

Where the paper says it

During treatment planning, clinically established dose constraints derived from DVH metrics, such as mean lung dose (MLD), V5 (lung volume receiving ≥ 5Gy), V20 (lung volume receiving ≥ 20Gy), and V30 (lung volume receiving ≥ 30Gy), are evaluated for toxicity risk assessment. Building upon these dosimetric indices, normal tissue complication probability (NTCP) models, such as the Lyman-Kutcher-Burman (LKB) model [6–8], have become widely used to estimate the probability of radiation-induced lung injury [9, 10].

T. Complication probability as assessed from dose-volume histograms. p. 0
Where the paper says it

org; Abstract Radiation pneumonitis (RP) is a common and clinically significant toxicity of thoracic radiation therapy that can cause pulmonary morbidity and impair quality of life. Although conventional dose-volume histogram–based metrics and normal tissue complication probability models are widely used for RP risk assessment, they inadequately capture the complex spatial, anatomical, and patient-specific factors underlying radiation-induced lung injury.

[7] Kutcher, G. p. 1

Where the paper says it

During treatment planning, clinically established dose constraints derived from DVH metrics, such as mean lung dose (MLD), V5 (lung volume receiving ≥ 5Gy), V20 (lung volume receiving ≥ 20Gy), and V30 (lung volume receiving ≥ 30Gy), are evaluated for toxicity risk assessment. Building upon these dosimetric indices, normal tissue complication probability (NTCP) models, such as the Lyman-Kutcher-Burman (LKB) model [6–8], have become widely used to estimate the probability of radiation-induced lung injury [9, 10].

J. & Burman, C. p. 1
Where the paper says it

During treatment planning, clinically established dose constraints derived from DVH metrics, such as mean lung dose (MLD), V5 (lung volume receiving ≥ 5Gy), V20 (lung volume receiving ≥ 20Gy), and V30 (lung volume receiving ≥ 30Gy), are evaluated for toxicity risk assessment. Building upon these dosimetric indices, normal tissue complication probability (NTCP) models, such as the Lyman-Kutcher-Burman (LKB) model [6–8], have become widely used to estimate the probability of radiation-induced lung injury [9, 10].

Calculation of complication probability factors for non-uniform normal tissue irradiation: The effective volume method gerald. p. 0
Where the paper says it

org; Abstract Radiation pneumonitis (RP) is a common and clinically significant toxicity of thoracic radiation therapy that can cause pulmonary morbidity and impair quality of life. Although conventional dose-volume histogram–based metrics and normal tissue complication probability models are widely used for RP risk assessment, they inadequately capture the complex spatial, anatomical, and patient-specific factors underlying radiation-induced lung injury.

[8] Kutcher, G. p. 1

Where the paper says it

During treatment planning, clinically established dose constraints derived from DVH metrics, such as mean lung dose (MLD), V5 (lung volume receiving ≥ 5Gy), V20 (lung volume receiving ≥ 20Gy), and V30 (lung volume receiving ≥ 30Gy), are evaluated for toxicity risk assessment. Building upon these dosimetric indices, normal tissue complication probability (NTCP) models, such as the Lyman-Kutcher-Burman (LKB) model [6–8], have become widely used to estimate the probability of radiation-induced lung injury [9, 10].

, Burman, C. p. 1
Where the paper says it

During treatment planning, clinically established dose constraints derived from DVH metrics, such as mean lung dose (MLD), V5 (lung volume receiving ≥ 5Gy), V20 (lung volume receiving ≥ 20Gy), and V30 (lung volume receiving ≥ 30Gy), are evaluated for toxicity risk assessment. Building upon these dosimetric indices, normal tissue complication probability (NTCP) models, such as the Lyman-Kutcher-Burman (LKB) model [6–8], have become widely used to estimate the probability of radiation-induced lung injury [9, 10].

, Brewster, L., Goitein, M. & Mohan, R. Histogram reduction method for calculating complication probabilities for three-dimensional treatment planning evaluations. p. 1
Where the paper says it

During treatment planning, clinically established dose constraints derived from DVH metrics, such as mean lung dose (MLD), V5 (lung volume receiving ≥ 5Gy), V20 (lung volume receiving ≥ 20Gy), and V30 (lung volume receiving ≥ 30Gy), are evaluated for toxicity risk assessment. Building upon these dosimetric indices, normal tissue complication probability (NTCP) models, such as the Lyman-Kutcher-Burman (LKB) model [6–8], have become widely used to estimate the probability of radiation-induced lung injury [9, 10].

[9] He, Y. et al. Quantifying the effect of 4-dimensional computed tomography–based deformable dose accumulation on representing radiation damage for patients with locally advanced non-small cell lung cancer treated with standard-fractionated intensity-modulated radiation therapy. p. 3

Where the paper says it

The first cohort is derived from the NRG Radiation Therapy Oncology Group (RTOG) 0617 randomized clinical trial and accessed through The Cancer Imaging Archive [34] with approval from the National Cancer Institute (NCI) National Clinical Trials Network (NCTN)/National Community Oncology Research Program (NCORP). The trial includes 544 patients with stage III non-small-cell lung cancer (NSCLC) who were enrolled in a randomized study evaluating radiation dose escalation in combination of chemotherapy, with and without cetuximab [35, 36].

[10] Liu, Y. p. 0

Where the paper says it

Multimodal Deep Learning for Uncertainty-Aware Radiation Pneumonitis Risk Prediction Jin Yang, Tian Liu, Jing Wang, Robert Samstein, Kenneth Rosenzweig, Julie Bloom, Ming Chao* Department of Radiation Oncology, Icahn School of Medicine at Mount Sinai, New York, NY 10029, USA. *Corresponding author(s).

et al. Relative biological effectiveness derived from ntcp analysis of carbon ion vs. p. 1
Where the paper says it

During treatment planning, clinically established dose constraints derived from DVH metrics, such as mean lung dose (MLD), V5 (lung volume receiving ≥ 5Gy), V20 (lung volume receiving ≥ 20Gy), and V30 (lung volume receiving ≥ 30Gy), are evaluated for toxicity risk assessment. Building upon these dosimetric indices, normal tissue complication probability (NTCP) models, such as the Lyman-Kutcher-Burman (LKB) model [6–8], have become widely used to estimate the probability of radiation-induced lung injury [9, 10].

photon radiotherapy in locally advanced non-small cell lung cancer. p. 3
Where the paper says it

The first cohort is derived from the NRG Radiation Therapy Oncology Group (RTOG) 0617 randomized clinical trial and accessed through The Cancer Imaging Archive [34] with approval from the National Cancer Institute (NCI) National Clinical Trials Network (NCTN)/National Community Oncology Research Program (NCORP). The trial includes 544 patients with stage III non-small-cell lung cancer (NSCLC) who were enrolled in a randomized study evaluating radiation dose escalation in combination of chemotherapy, with and without cetuximab [35, 36].

[11] Marks, L. B. et al. Radiation dose–volume effects in the lung. p. 0

Where the paper says it

org; Abstract Radiation pneumonitis (RP) is a common and clinically significant toxicity of thoracic radiation therapy that can cause pulmonary morbidity and impair quality of life. Although conventional dose-volume histogram–based metrics and normal tissue complication probability models are widely used for RP risk assessment, they inadequately capture the complex spatial, anatomical, and patient-specific factors underlying radiation-induced lung injury.

[12] Van den Bosch, L. et al. Key challenges in normal tissue complication probability model development and validation: towards a comprehensive strategy. p. 1

Where the paper says it

During treatment planning, clinically established dose constraints derived from DVH metrics, such as mean lung dose (MLD), V5 (lung volume receiving ≥ 5Gy), V20 (lung volume receiving ≥ 20Gy), and V30 (lung volume receiving ≥ 30Gy), are evaluated for toxicity risk assessment. Building upon these dosimetric indices, normal tissue complication probability (NTCP) models, such as the Lyman-Kutcher-Burman (LKB) model [6–8], have become widely used to estimate the probability of radiation-induced lung injury [9, 10].

[13] Valdes, G., Solberg, T. D., Heskel, M., Ungar, L. & Simone, C. B. Using machine learning to predict radiation pneumonitis in patients with stage i non-small cell lung cancer treated with stereotactic body radiation therapy. p. 3

Where the paper says it

The first cohort is derived from the NRG Radiation Therapy Oncology Group (RTOG) 0617 randomized clinical trial and accessed through The Cancer Imaging Archive [34] with approval from the National Cancer Institute (NCI) National Clinical Trials Network (NCTN)/National Community Oncology Research Program (NCORP). The trial includes 544 patients with stage III non-small-cell lung cancer (NSCLC) who were enrolled in a randomized study evaluating radiation dose escalation in combination of chemotherapy, with and without cetuximab [35, 36].

[14] Krafft, S. P. et al. The utility of quantitative ct radiomics features for improved prediction of radiation pneumonitis. p. 0

Where the paper says it

Although conventional dose-volume histogram–based metrics and normal tissue complication probability models are widely used for RP risk assessment, they inadequately capture the complex spatial, anatomical, and patient-specific factors underlying radiation-induced lung injury. Recent machine learning approaches have improved RP risk prediction by integrating multimodal clinical and imaging information; however, most provide a point risk estimate without quantifying the reliability of individual predictions, limiting their potential clinical utility.

[15] Yu, H. et al. Machine learning to build and validate a model for radiation pneumonitis prediction in patients with non small cell lung cancer. p. 3

Where the paper says it

The first cohort is derived from the NRG Radiation Therapy Oncology Group (RTOG) 0617 randomized clinical trial and accessed through The Cancer Imaging Archive [34] with approval from the National Cancer Institute (NCI) National Clinical Trials Network (NCTN)/National Community Oncology Research Program (NCORP). The trial includes 544 patients with stage III non-small-cell lung cancer (NSCLC) who were enrolled in a randomized study evaluating radiation dose escalation in combination of chemotherapy, with and without cetuximab [35, 36].

[16] Luna, J. M. et al. Predicting radiation pneumonitis in locally advanced stage ii–iii non-small cell lung cancer using machine learning. p. 3

Where the paper says it

The first cohort is derived from the NRG Radiation Therapy Oncology Group (RTOG) 0617 randomized clinical trial and accessed through The Cancer Imaging Archive [34] with approval from the National Cancer Institute (NCI) National Clinical Trials Network (NCTN)/National Community Oncology Research Program (NCORP). The trial includes 544 patients with stage III non-small-cell lung cancer (NSCLC) who were enrolled in a randomized study evaluating radiation dose escalation in combination of chemotherapy, with and without cetuximab [35, 36].

[17] Hirose, T.-a. et al. Radiomic prediction of radiation pneumonitis on pretreatment planning computed tomography images prior to lung cancer stereotactic body radiation therapy. p. 1

Where the paper says it

This framework may ultimately support more informed individualized treatment planning and safer clinical decision-making in thoracic radiation therapy. Keywords: Radiation Pneumonitis, Self-supervised Multimodal Learning, Bayesian Approximation, Predictive Uncertainty, Thoracic Radiation Therapy 1 Introduction Radiation pneumonitis (RP) is a common and clinically significant toxicity of thoracic radiation therapy (RT) for lung cancer and other thoracic malignancies and represents an important constraint on the safe delivery of radiation dose [1, 2].

[18] Feng, A. et al. Improvement of prediction performance for radiation pneumonitis by using 3-dimensional dosiomic features. p. 5

Where the paper says it

CT Dose distribution Multi-modal features 𝐹 Bayesian classifier True Predictive distribution Encoder Encoder Latent space PoE/MoE CT 𝒩(𝜇1, 𝜎1 2𝑰) 𝒩(𝜇2, 𝜎2 2𝑰) Dose distribution 𝒩(𝜇, 𝜎 2𝑰) 𝑓𝜃1(Θ) 𝑓𝜃2(Θ) Decoder Decoder 𝑔𝜃1(Θ) 𝑔𝜃2(Θ) Reconstructed CT Reconstructed Dose FC ReLU MC Dropout FC RP Risk Prediction Aleatoric Uncertainty Epistemic Uncertainty Label Noise Transition Layer False positive rate False negative rate Observed Predictive distribution Label Uncertainty Modality Reconstruction ℒ𝑅𝑒𝑐 Weighted Binary Cross Entropy Loss ℒ𝑊𝐵𝐶𝐸 RP grade Diffusion Transformer (DiT) Noise Diffusion Loss ℒ𝐷𝑖𝑓𝑓 Timestep t Self-supervised pre -training Supervised Fine -tuning Fig. 1: Multimodal Bayesian Diffusion Transformer (MM-DiT) framework for radiation pneumonitis risk prediction and uncertainty quantification.

[19] Gadsby, A. et al. Impact of normal lung volume choices on radiation pneumonitis risk prediction in locally non-small cell lung cancer radiation therapy. p. 3

Where the paper says it

The first cohort is derived from the NRG Radiation Therapy Oncology Group (RTOG) 0617 randomized clinical trial and accessed through The Cancer Imaging Archive [34] with approval from the National Cancer Institute (NCI) National Clinical Trials Network (NCTN)/National Community Oncology Research Program (NCORP). The trial includes 544 patients with stage III non-small-cell lung cancer (NSCLC) who were enrolled in a randomized study evaluating radiation dose escalation in combination of chemotherapy, with and without cetuximab [35, 36].

[20] Cui, S., Ten Haken, R. K. & El Naqa, I. Integrating multiomics information in deep learning architectures for joint actuarial outcome prediction in non-small cell lung cancer patients after radiation therapy. p. 3

Where the paper says it

The first cohort is derived from the NRG Radiation Therapy Oncology Group (RTOG) 0617 randomized clinical trial and accessed through The Cancer Imaging Archive [34] with approval from the National Cancer Institute (NCI) National Clinical Trials Network (NCTN)/National Community Oncology Research Program (NCORP). The trial includes 544 patients with stage III non-small-cell lung cancer (NSCLC) who were enrolled in a randomized study evaluating radiation dose escalation in combination of chemotherapy, with and without cetuximab [35, 36].

[21] Jiang, W., Song, Y., Sun, Z., Qiu, J. & Shi, L. Dosimetric factors and radiomics features within different regions of interest in planning ct images for improving the prediction of radiation pneumonitis. p. 1

Where the paper says it

Furthermore, RP is a multifactorial process influenced not only by radiation dose but also by patient-specific characteristics, pulmonary function, tumor location, anatomical imaging features, and individual radiosensitivity [11, 12]. Consequently, patients with nearly identical DVH metrics can experience markedly different clinical outcomes, underscoring the limited ability of traditional dosimetric models to capture the complex biological and patient-specific factors underlying radiation-induced lung injury.

[22] Jiao, Y. et al. Integrating radiomics and dosiomics with lung biologically equivalent dose for predicting symptomatic radiation pneumonitis after lung sbrt: A dual-center study. p. 0

Where the paper says it

Although conventional dose-volume histogram–based metrics and normal tissue complication probability models are widely used for RP risk assessment, they inadequately capture the complex spatial, anatomical, and patient-specific factors underlying radiation-induced lung injury. Recent machine learning approaches have improved RP risk prediction by integrating multimodal clinical and imaging information; however, most provide a point risk estimate without quantifying the reliability of individual predictions, limiting their potential clinical utility.

[23] Reuter, L. M. et al. Prediction of symptomatic radiation pneumonitis in lung cancer patients: A radiomics and dosiomics machine learning approach using the prospective multicenter rtog 0617 and requite trials. p. 3

Where the paper says it

The first cohort is derived from the NRG Radiation Therapy Oncology Group (RTOG) 0617 randomized clinical trial and accessed through The Cancer Imaging Archive [34] with approval from the National Cancer Institute (NCI) National Clinical Trials Network (NCTN)/National Community Oncology Research Program (NCORP). The trial includes 544 patients with stage III non-small-cell lung cancer (NSCLC) who were enrolled in a randomized study evaluating radiation dose escalation in combination of chemotherapy, with and without cetuximab [35, 36].

[24] Ovadia, Y. et al. Can you trust your model’s uncertainty? p. 0

Where the paper says it

The resulting representations are further refined using a latent diffusion transformer and transferred to a Bayesian prediction framework for probabilistic RP risk estimation. A learnable label-noise model is incorporated to explicitly account for uncertainty arising from imperfect toxicity annotations.

evaluating predictive uncertainty under dataset shift. p. 3
Where the paper says it

The first cohort is derived from the NRG Radiation Therapy Oncology Group (RTOG) 0617 randomized clinical trial and accessed through The Cancer Imaging Archive [34] with approval from the National Cancer Institute (NCI) National Clinical Trials Network (NCTN)/National Community Oncology Research Program (NCORP). The trial includes 544 patients with stage III non-small-cell lung cancer (NSCLC) who were enrolled in a randomized study evaluating radiation dose escalation in combination of chemotherapy, with and without cetuximab [35, 36].

[25] Loftus, T. J. et al. Uncertainty-aware deep learning in healthcare: a scoping review. p. 0

Where the paper says it

Multimodal Deep Learning for Uncertainty-Aware Radiation Pneumonitis Risk Prediction Jin Yang, Tian Liu, Jing Wang, Robert Samstein, Kenneth Rosenzweig, Julie Bloom, Ming Chao* Department of Radiation Oncology, Icahn School of Medicine at Mount Sinai, New York, NY 10029, USA. *Corresponding author(s).

[26] Yang, S. p. 0

Where the paper says it

Multimodal Deep Learning for Uncertainty-Aware Radiation Pneumonitis Risk Prediction Jin Yang, Tian Liu, Jing Wang, Robert Samstein, Kenneth Rosenzweig, Julie Bloom, Ming Chao* Department of Radiation Oncology, Icahn School of Medicine at Mount Sinai, New York, NY 10029, USA. *Corresponding author(s).

et al. A parametrical model for instance-dependent label noise. p. 0
Where the paper says it

The resulting representations are further refined using a latent diffusion transformer and transferred to a Bayesian prediction framework for probabilistic RP risk estimation. A learnable label-noise model is incorporated to explicitly account for uncertainty arising from imperfect toxicity annotations.

[27] Wei, Y. et al. Deep learning with noisy labels in medical prediction problems: a scoping review. p. 5

Where the paper says it

MM-DiT is a multimodal deep learning framework that performs RP prediction through two sequential stages: self-supervised multimodal representation learning and supervised Bayesian fine-tuning. During self-supervised pre-training, a multi-modal Diffusion Transformer (DiT) learns cross-modal representations from planning CT images and 3D dose distributions without using RP labels.

[28] Tyralis, H. & Papacharalampous, G. A review of predictive uncertainty estimation with machine learning. p. 0

Where the paper says it

Recent machine learning approaches have improved RP risk prediction by integrating multimodal clinical and imaging information; however, most provide a point risk estimate without quantifying the reliability of individual predictions, limiting their potential clinical utility. We propose a Multimodal Bayesian Diffusion Transformer (MM-DiT) framework that jointly estimates RP risk and characterizes the sources of predictive uncertainty.

[29] Stutz, D. et al. Evaluating medical ai systems in dermatology under uncertain ground truth. p. 5

Where the paper says it

A Diffusion Transformer (DiT) is subsequently employed to learn robust multimodal representations from the fused latent distributions. During self-supervised pre-training, the encoders and DiT are optimized without access to ground truth RP labels, enabling the model to learn disease-agnostic anatomical and 6

[30] Kurz, A. et al. Uncertainty estimation in medical image classification: systematic review. p. 2

Where the paper says it

Despite the growing recognition of uncertainty as an important component of trustworthy medical AI [30, 31], uncertainty-aware approaches remain largely unexplored in RP prediction [32, 33]. RP risk estimation is inherently subject to multiple sources of uncertainty, including variability in patient-specific susceptibility and the complex relationship between radiation exposure and lung injury, limited information available to the predictive model, ambiguity or errors in clinical toxicity assessment.

[31] Huang, L., Ruan, S., Xing, Y. & Feng, M. A review of uncertainty quantification in medical image analysis: Probabilistic and non-probabilistic methods. p. 3

Where the paper says it

calibrated probabilistic RP risk estimates together with complementary measures of aleatoric, epistemic, and label uncertainty, enabling assessment of prediction reliability and identification of cases that may warrant additional clinical evaluation. We evaluate MM-DiT in two independent cohorts using complementary assessments of predictive discrimination, calibration, and uncertainty quantification.

[32] Wahid, K. A. et al. Artificial intelligence uncertainty quantification in radiotherapy applications- a scoping review. p. 2

Where the paper says it

To overcome these limitations, recent studies have increasingly explored artificial intelligence (AI), particularly machine learning (ML) and deep learning (DL), for RP risk prediction [13–19]. Unlike conventional NTCP models, these approaches can integrate heterogeneous sources of information, including patient demographics, clinical characteristics, RT dose distributions, and radiomic features extracted from imaging such as computed tomography (CT).

[33] D¨unger, L., M¨ausel, E., Zwanenburg, A. & L¨ock, S. Uncertainties in outcome modelling in radiation oncology. p. 2

Where the paper says it

MM-DiT integrates planning CT images and 3D radiation dose distributions to estimate both an individual patient’s probability of developing RP and the uncertainty associated with that prediction. Rather than relying solely on supervised learning from limited toxicity labels, MM-DiT first learns multimodal representations through self-supervised pre-training using routinely acquired planning data, reducing dependence on potentially noisy outcome labels and promoting robust representation learning.

Pruning redundant reasoning tokens

The authors propose MM-DiT to provide individualized RP probabilities together with complementary measures of aleatoric, epistemic, and label uncertainty. This framework distinguishes between what it predicts and how certain that prediction is. p. 0

Where the paper says it

Recent machine learning approaches have improved RP risk prediction by integrating multimodal clinical and imaging information; however, most provide a point risk estimate without quantifying the reliability of individual predictions, limiting their potential clinical utility. We propose a Multimodal Bayesian Diffusion Transformer (MM-DiT) framework that jointly estimates RP risk and characterizes the sources of predictive uncertainty.

The decomposition of predictive uncertainty provides additional insight into the factors that limit RP risk prediction. p. 3
Where the paper says it

calibrated probabilistic RP risk estimates together with complementary measures of aleatoric, epistemic, and label uncertainty, enabling assessment of prediction reliability and identification of cases that may warrant additional clinical evaluation. We evaluate MM-DiT in two independent cohorts using complementary assessments of predictive discrimination, calibration, and uncertainty quantification.

How the training signal is built

The model first learns multimodal representations through self-supervised pre-training using routinely acquired planning data, reducing dependence on potentially noisy outcome labels. This stage utilizes a multi-modal Diffusion Transformer (DiT) to learn cross-modal representations from planning CT images and 3D dose distributions without using RP labels. p. 5

Where the paper says it

MM-DiT is a multimodal deep learning framework that performs RP prediction through two sequential stages: self-supervised multimodal representation learning and supervised Bayesian fine-tuning. During self-supervised pre-training, a multi-modal Diffusion Transformer (DiT) learns cross-modal representations from planning CT images and 3D dose distributions without using RP labels.

During supervised fine-tuning, the observed RP labels are treated as potentially noisy measurements of an unobserved true toxicity outcome rather than error-free ground truth labels. p. 2
Where the paper says it

MM-DiT integrates planning CT images and 3D radiation dose distributions to estimate both an individual patient’s probability of developing RP and the uncertainty associated with that prediction. Rather than relying solely on supervised learning from limited toxicity labels, MM-DiT first learns multimodal representations through self-supervised pre-training using routinely acquired planning data, reducing dependence on potentially noisy outcome labels and promoting robust representation learning.

The classifier is fine-tuned using a warm-up training scheme with a weighted binary cross-entropy loss and subsequently a focal loss for supervised fine-tuning. p. 5
Where the paper says it

CT Dose distribution Multi-modal features 𝐹 Bayesian classifier True Predictive distribution Encoder Encoder Latent space PoE/MoE CT 𝒩(𝜇1, 𝜎1 2𝑰) 𝒩(𝜇2, 𝜎2 2𝑰) Dose distribution 𝒩(𝜇, 𝜎 2𝑰) 𝑓𝜃1(Θ) 𝑓𝜃2(Θ) Decoder Decoder 𝑔𝜃1(Θ) 𝑔𝜃2(Θ) Reconstructed CT Reconstructed Dose FC ReLU MC Dropout FC RP Risk Prediction Aleatoric Uncertainty Epistemic Uncertainty Label Noise Transition Layer False positive rate False negative rate Observed Predictive distribution Label Uncertainty Modality Reconstruction ℒ𝑅𝑒𝑐 Weighted Binary Cross Entropy Loss ℒ𝑊𝐵𝐶𝐸 RP grade Diffusion Transformer (DiT) Noise Diffusion Loss ℒ𝐷𝑖𝑓𝑓 Timestep t Self-supervised pre -training Supervised Fine -tuning Fig. 1: Multimodal Bayesian Diffusion Transformer (MM-DiT) framework for radiation pneumonitis risk prediction and uncertainty quantification.

Where the gains come from

The CNN+PoE configuration achieved the highest overall performance across both cohorts, with AUC values reaching 85.59% in the RTOG 0617 cohort and 89.11% in the MSHS cohort. The superior discriminative ability of MM-DiT is demonstrated by its higher AUC compared to single-modality models and other multimodal VAEs. The framework supports risk-based decision-making across a range of clinically relevant thresholds, showing a positive net benefit in decision curve analysis. p. 3

Where the paper says it

We evaluate MM-DiT in two independent cohorts using complementary assessments of predictive discrimination, calibration, and uncertainty quantification. Clinical utility is further evaluated using decision curve analysis to determine whether incorporating uncertainty-aware RP risk estimates could provide greater net benefit across clinically relevant decision thresholds.

The joint consideration of predicted risk and predictive uncertainty provides a framework for clinically interpreting model outputs. This leads to four risk–uncertainty groups, such as High Risk + High Confidence, which suggests immediate clinical intervention. p. 1

Where the paper says it

Preventing RP while maintaining optimal treatment outcomes therefore remains a central challenge in thoracic RT. Accurate identification of patients at high risk for RP before treatment could facilitate early risk stratification and more informed clinical management, providing a foundation for individualized treatment decisions and improved patient outcomes.

This framework extends conventional risk stratification by incorporating the reliability of each prediction into its interpretation. p. 3
Where the paper says it

By combining individualized RP risk prediction with explicit characterization of aleatoric, epistemic, and label uncertainty, MM-DiT provides not only an estimate of RP risk but also information regarding the reliability of that estimate. This uncertainty-aware framework may provide a more informative basis for risk stratification and clinical decision-making, supporting the potential translation of AI-assisted RP prediction into routine thoracic radiation oncology.

The present study was retrospective and relied on pretreatment CT and radiation dose distributions as the primary model inputs. The predominance of aleatoric uncertainty observed in the analysis suggests the need for future studies incorporating toxicity-rich longitudinal data acquired during and after RT. Furthermore, additional multi-institutional validation across diverse patient populations and imaging protocols is needed to establish generalizability. p. 1

Where the paper says it

First, DVHs compress complex threedimensional (3D) dose distributions into one-dimensional summary statistics, thereby discarding spatial information regarding regional dose deposition and tissue heterogeneity. Furthermore, RP is a multifactorial process influenced not only by radiation dose but also by patient-specific characteristics, pulmonary function, tumor location, anatomical imaging features, and individual radiosensitivity [11, 12].

The risk–uncertainty framework is not intended to prescribe specific treatment modifications based solely on model output but provides a mechanism for identifying predictions that may be sufficiently supported to inform clinical consideration. p. 2
Where the paper says it

Consequently, even when a model demonstrates good overall discrimination, its output for a specific patient may not indicate how strongly that prediction is supported by the available evidence. This limitation is particularly important when RP risk estimates are used to inform treatment planning or clinical management, where the reliability of an individual prediction may be as important as its estimated probability [24–29].

In this study, MM-DiT developed a Bayesian multimodal framework that integrates planning CT imaging and 3D radiation dose distributions for individualized RP risk prediction and explicit uncertainty quantification. This framework provides a more transparent and reliable approach to AI-assisted toxicity assessment. p. 1

Where the paper says it

representation learning, Bayesian uncertainty quantification, and explicit modeling of label noise, MM-DiT offers a more reliable and transparent approach to AI-assisted RP risk assessment. This framework may ultimately support more informed individualized treatment planning and safer clinical decision-making in thoracic radiation therapy.

Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.

Improvements for AI systems

  1. Multimodal representation learning via self-supervised pre-training using a Diffusion Transformer (DiT) learns disease agnostic anatomical and dosimetric representations without relying on potentially noisy toxicity labels, thereby reducing dependence on limited outcome labels.

  2. Bayesian prediction framework with Monte Carlo Dropout approximates the posterior distribution over model parameters, generating a distribution of RP risk predictions rather than a single point estimate for each patient.

  3. Explicit modeling of uncertainty decomposes total predictive uncertainty into aleatoric, epistemic, and label uncertainty, allowing clinicians to distinguish between inherent data noise, model limitations due to insufficient training data (epistemic), and inconsistencies in toxicity annotations (label).

  4. The framework provides a clinical interpretation framework by stratifying patients into four risk-uncertainty groups—Low Risk High Confidence, Low Risk Low Confidence, High Risk High Confidence, and High Risk Low Confidence—to guide decisions like standard treatment or immediate intervention.

Abstract

Radiation pneumonitis (RP) is a potentially serious complication of thoracic radiotherapy, making reliable risk prediction important for identifying high-risk patients and guiding effective toxicity management. Although deep learning models can predict RP risk, uncertainty quantification remains underdeveloped in radiation toxicity assessment. We developed a Multimodal Bayesian Diffusion Transformer (MM-DiT) that integrates pretreatment computed tomography (CT) and three-dimensional planning dose distributions for RP risk prediction and uncertainty quantification. MM-DiT combines self-supervised multimodal representation learning with Bayesian classification to estimate individual risk and characterize aleatoric, epistemic, and label uncertainty. In independent Mount Sinai Health System (MSHS) and RTOG cohorts, MM-DiT achieved areas under the receiver operating characteristic curve (AUCs) of 0.89 and 0.86, respectively, with expected calibration errors of 0.02 and 0.03. Incorporating post-treatment CT increased AUC to 0.95 in MSHS and reduced aleatoric and epistemic uncertainty by approximately 40-45% and 40%, respectively. These findings establish a framework for uncertainty-aware radiation toxicity prediction that complements risk estimates with information about predictive reliability. By integrating multimodal learning and uncertainty quantification, MM-DiT may advance more interpretable and trustworthy AI-assisted toxicity assessment and support individualized radiotherapy planning.

Related papers