Multimodal Deep Learning for Uncertainty-Aware Radiation Pneumonitis Risk Prediction
Listen
Radio episode about this paper
Transcript
Introduction to the show: ident: Electrical Engineering. The day in electrical engineering: signals, systems and the hardware that carries them.
Rosa: We're looking at the paper: "Multimodal Deep Learning for Uncertainty-Aware Radiation Pneumonitis Risk Prediction".
Dev: Here's the gist of the paper. Radiation pneumonitis (RP) is a common and clinically significant toxicity of thoracic radiation therapy that can cause pulmonary morbidity and impair quality of life.
Rosa: MM-DiT provides individualized RP risk estimates together with complementary measures of aleatoric, epistemic, and label uncertainty.
Dev: Problem Statement Conventional dose-volume histogram–based metrics and normal tissue complication probability models are widely used for RP risk assessment but inadequately capture the complex spatial, anatomical, and patient-specific factors underlying radiation-induced lung injury.
Rosa: These traditional approaches compress complex three-dimensional (3D) dose distributions into one-dimensional summary statistics, thereby discarding spatial information regarding regional dose deposition and tissue heterogeneity.
Paper summary: Rosa: So, to get into what this paper is really saying, they start by pointing out that conventional dose-volume histogram based metrics and normal tissue complication probability models just can't handle the complexity of radiation lung injury >
Dev: They argue these traditional models struggle because they compress everything into one dimension when there's so much spatial information about the dose distribution that gets lost >
Rosa: The thesis here is that by using a Multimodal Bayesian Diffusion Transformer, they can jointly estimate RP risk and characterize the sources of predictive uncertainty >
Taro: Why does characterizing uncertainty matter for us? Because right now, most AI methods just give you a single number—a point estimate—without telling you how much to trust it >
Dev: The paper claims that this approach offers a more reliable and transparent way to do AI assisted RP risk assessment by providing that explicit modeling of label noise >
Rosa: They’re trying to support more informed individualized treatment planning and safer clinical decision making in thoracic radiation therapy using this framework >
Taro: It's about giving clinicians a tool where they can stratify patients into risk uncertainty groups, like high risk plus high confidence, which guides immediate intervention decisions >
Dev: The paper is essentially showing how integrating heterogeneous sources of information—patient demographics, clinical characteristics, dose distributions, and radiomic features from imaging—can capture the complex interactions contributing to RP development >
Rosa: It’s about moving away from just a deterministic prediction to something that acknowledges the reliability of every individual prediction >
Taro: That's what makes it relevant because in real-world clinical settings, you can't afford to make a decision without knowing the risk and how certain you are about that risk >
Dev: So, this framework is designed to be more reliable and transparent for those who need to use AI in their treatment planning process >
Rosa: The authors of this paper are Jin Yang, Tian Liu, Jing Wang, Robert Samstein, Kenneth Rosenzweig, Julie Bloom and Ming Chao at the Department of Radiation Oncology at Icahn School of Medicine at Mount Sinai in New York City >
Taro: It’s interesting that they’re focusing on lung cancer and other thoracic malignancies as the primary context for this work because RP is such a significant constraint on safe dose delivery >
Dev: The paper also notes that the clinical translation of AI methods remains limited by challenges like generalizability, calibration, interpretation, and reliable assessment of individual patient risk >
Rosa: So, even with improved performance numbers like AUC values reaching eighty-five point five nine percent in the RTOG six hundred seventeen cohort and eighty-nine point one one percent in the MSHS cohort >
Taro: Those performance numbers are good, but they're just one piece of a puzzle when you consider how robust the framework handles those uncertainties when the world misbehaves >
Dev: It shows that integrating multimodal information can improve predictive performance over models that only look at one modality, like just dose distribution or just imaging features >
Rosa: So, this paper is essentially proposing a way to get better predictions by making sure we also understand the uncertainty around those predictions >
Conclusion: Taro: To wrap this up, the Multimodal Deep Learning for Uncertainty-Aware Radiation Pneumonitis Risk Prediction framework is a Bayesian multimodal approach that integrates planning CT imaging and three dee radiation dose distributions for individualized RP risk prediction and explicit uncertainty quantification > <ref:2608.28937#pg2>
Dev: It's about providing a more transparent and reliable approach to AI assisted RP risk assessment by giving us that explicit modeling of label noise >
Rosa: It moves us toward more informed individualized treatment planning because it allows clinicians to see not just the predicted risk but also how certain the model is in its own assessment >
Taro: This framework extends conventional risk stratification toward uncertainty-aware clinical interpretation, allowing you to categorize patients into groups like high risk plus high confidence for immediate intervention considerations >
Dev: It’s a tool that helps identify predictions that are sufficiently supported to inform clinical consideration without necessarily prescribing specific treatment modifications based solely on the model output >
Taro: It’s a framework that gives us a mechanism for identifying predictions that may be sufficiently supported to inform clinical consideration >
Dev: Ultimately, it's about providing a more transparent and reliable approach to AI assisted RP risk assessment >
Jin Yang, Tian Liu, Jing Wang, Robert Samstein, Kenneth Rosenzweig, Julie Bloom
Department of Radiation Oncology, Icahn School of Medicine at Mount Sinai
eess.IV
Submitted: 2026-08-28
Updated: 2026-10-31
Comments: 28 pages, 9 figures, 5 tables
License: http://creativecommons.org/licenses/by/4.0/
Importance score: 92/100
The gist: Radiation pneumonitis (RP) is a common and clinically significant toxicity of thoracic radiation therapy that can cause pulmonary morbidity and impair quality of life.
Key concepts
- Multimodal Bayesian Diffusion Transformer (MM-DiT)
- This is the core framework that integrates planning CT images and 3D dose distributions. It uses a diffusion transformer structure to learn complex, cross-modal representations from these different data types without needing prior toxicity labels. This allows the model to understand how anatomy and dose interact.
- Aleatoric Uncertainty (Ualea)
- This uncertainty reflects the inherent randomness or stochasticity present in the input data itself. It captures how noisy or variable the patient's characteristics are, which directly impacts the prediction's variance. High aleatoric uncertainty means the model is inherently less certain about that specific prediction.
- Epistemic Uncertainty (Uepst)
- This uncertainty measures limitations in the model itself, stemming from insufficient or unrepresentative training data. It indicates that a patient's characteristics fall outside what the model has seen before. High epistemic uncertainty suggests the model is unsure because it hasn't encountered similar cases.
- Label Uncertainty (Ulabel)
- This component explicitly models inconsistencies and subjectivity in the toxicity annotations used for training. It quantifies how much the true label might fluctuate based on the predicted risk, providing a measure of confidence in the ground truth data.
Terminology
Summary
Radiation pneumonitis (RP) is a common and clinically significant toxicity of thoracic radiation therapy that can cause pulmonary morbidity and impair quality of life. p. 0 chao@mountsinai. org;
Abstract
Radiation pneumonitis (RP) is a common and clinically significant toxicity of
thoracic radiation therapy that can cause pulmonary morbidity and impair quality of life. A learnable label-noise model is incorporated
to explicitly account for uncertainty arising from imperfect toxicity annotations. MM-DiT, therefore, provides individualized RP risk estimates together with
complementary measures of aleatoric, epistemic, and label uncertainty, enabling
assessment of prediction reliability at the individual-patient level.Where the paper says it
Where the paper says it
Problem Statement
Conventional dose-volume histogram–based metrics and normal tissue complication probability models are widely used for RP risk assessment but inadequately capture the complex spatial, anatomical, and patient-specific factors underlying radiation-induced lung injury. These traditional approaches compress complex three-dimensional (3D) dose distributions into one-dimensional summary statistics, thereby discarding spatial information regarding regional dose deposition and tissue heterogeneity. p. 1 While clinically established,
these approaches have important limitations. First, DVHs compress complex threedimensional (3D) dose distributions into one-dimensional summary statistics, thereby
discarding spatial information regarding regional dose deposition and tissue heterogeneity. First, DVHs compress complex threedimensional (3D) dose distributions into one-dimensional summary statistics, thereby
discarding spatial information regarding regional dose deposition and tissue heterogeneity. Furthermore, RP is a multifactorial process influenced not only by radiation
dose but also by patient-specific characteristics, pulmonary function, tumor location,
anatomical imaging features, and individual radiosensitivity [11, 12]. Although many studies have demonstrated improved predictive
performance, their clinical translation remains limited by challenges including generalizability, calibration, interpretation, and reliable assessment of individual patient
risk. A fundamental limitation of current RP prediction models is that they predominantly use deterministic supervised learning and provide a point estimate of RP risk
without quantifying the uncertainty associated with individual predictions.Where the paper says it
Where the paper says it
Where the paper says it
Framework Architecture
The proposed Multimodal Bayesian Diffusion Transformer (MM-DiT) framework integrates planning computed tomography (CT) images and three-dimensional radiation dose distributions through self-supervised multimodal pre-training. This framework jointly estimates RP risk and characterizes the sources of predictive uncertainty. p. 0 Recent machine
learning approaches have improved RP risk prediction by integrating multimodal
clinical and imaging information; however, most provide a point risk estimate
without quantifying the reliability of individual predictions, limiting their potential clinical utility. We propose a Multimodal Bayesian Diffusion Transformer
(MM-DiT) framework that jointly estimates RP risk and characterizes the
sources of predictive uncertainty. Patients from two cohorts are independently and randomly divided into training and internal test sets at a ratio of
80% : 20%. MM-DiT is a multimodal deep learning framework that performs RP
prediction through two sequential stages: self-supervised multimodal representation
learning and supervised Bayesian fine-tuning. MM-DiT is a multimodal deep learning framework that performs RP
prediction through two sequential stages: self-supervised multimodal representation
learning and supervised Bayesian fine-tuning. During self-supervised pre-training, a
multi-modal Diffusion Transformer (DiT) learns cross-modal representations from
planning CT images and 3D dose distributions without using RP labels. MM-DiT integrates planning computed tomography (CT) images and three-dimensional radiation dose distributions through
self-supervised multimodal pre-training, reducing reliance on limited and potentially noisy toxicity labels. The resulting representations are further refined using
a latent diffusion transformer and transferred to a Bayesian prediction framework
for probabilistic RP risk estimation.Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Uncertainty Quantification
MM-DiT provides complementary measures of aleatoric, epistemic, and label uncertainty. The total predictive uncertainty is decomposed into three distinct components: aleatoric (Ualea), epistemic (Uepst), and label uncertainty (Ulabel). p. 6 During inference, stochastic
forward passes generate a distribution of RP risk predictions rather than a single
point estimate, with the mean of the predictive distribution used as the final RP
risk estimate. Predictive uncertainty is decomposed into three components: aleatoric,
epistemic, and label uncertainty. During inference, stochastic
forward passes generate a distribution of RP risk predictions rather than a single
point estimate, with the mean of the predictive distribution used as the final RP
risk estimate. Predictive uncertainty is decomposed into three components: aleatoric,
epistemic, and label uncertainty. Detailed demographic and clinical
characteristics for these two cohorts are summarized in Table 1. 2 Data Preprocessing
The planning CT images and 3D dose distributions undergo a multi-step preprocessing
pipeline prior to model training. The resulting representations are further refined using
a latent diffusion transformer and transferred to a Bayesian prediction framework
for probabilistic RP risk estimation. A learnable label-noise model is incorporated
to explicitly account for uncertainty arising from imperfect toxicity annotations.Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Performance and Utility
Across both independent cohorts, CNN+PoE achieved the highest overall performance, with AUC values reaching 85.59% in the RTOG 0617 cohort and 89. p. 3 Each patient record
includes a planning CT image set and corresponding 3D spatial dose distribution. The
first cohort is derived from the NRG Radiation Therapy Oncology Group (RTOG)
0617 randomized clinical trial and accessed through The Cancer Imaging Archive [34]
with approval from the National Cancer Institute (NCI) National Clinical Trials Network (NCTN)/National Community Oncology Research Program (NCORP). Each patient record
includes a planning CT image set and corresponding 3D spatial dose distribution. The
first cohort is derived from the NRG Radiation Therapy Oncology Group (RTOG)
0617 randomized clinical trial and accessed through The Cancer Imaging Archive [34]
with approval from the National Cancer Institute (NCI) National Clinical Trials Network (NCTN)/National Community Oncology Research Program (NCORP). A total of 416 subjects are therefore included from the RTOG 0617 cohort. The second cohort comprised 213 patients with lung cancer who received thoracic RT
at the Mount Sinai Health System (MSHS). A total of 416 subjects are therefore included from the RTOG 0617 cohort. The second cohort comprised 213 patients with lung cancer who received thoracic RT
at the Mount Sinai Health System (MSHS). We evaluate MM-DiT in two independent cohorts using complementary assessments of predictive discrimination, calibration, and uncertainty quantification. Clinical
utility is further evaluated using decision curve analysis to determine whether incorporating uncertainty-aware RP risk estimates could provide greater net benefit across
clinically relevant decision thresholds. Each patient record
includes a planning CT image set and corresponding 3D spatial dose distribution. The
first cohort is derived from the NRG Radiation Therapy Oncology Group (RTOG)
0617 randomized clinical trial and accessed through The Cancer Imaging Archive [34]
with approval from the National Cancer Institute (NCI) National Clinical Trials Network (NCTN)/National Community Oncology Research Program (NCORP). A total of 416 subjects are therefore included from the RTOG 0617 cohort. The second cohort comprised 213 patients with lung cancer who received thoracic RT
at the Mount Sinai Health System (MSHS). We evaluate MM-DiT in two independent cohorts using complementary assessments of predictive discrimination, calibration, and uncertainty quantification. Clinical
utility is further evaluated using decision curve analysis to determine whether incorporating uncertainty-aware RP risk estimates could provide greater net benefit across
clinically relevant decision thresholds.Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Clinical Interpretation
The framework allows for stratifying patients into four risk–uncertainty groups based on predicted RP probability and overall uncertainty. This stratification includes categories such as High Risk + High Confidence, which suggests immediate clinical intervention. p. 1 Preventing RP while maintaining optimal treatment outcomes therefore remains a central challenge in thoracic RT. Accurate identification of patients at high risk for RP before treatment could facilitate early risk stratification and more informed clinical management, providing a
foundation for individualized treatment decisions and improved patient outcomes. Consequently, even when a model demonstrates good overall discrimination, its output for
a specific patient may not indicate how strongly that prediction is supported by the
available evidence. This limitation is particularly important when RP risk estimates
are used to inform treatment planning or clinical management, where the reliability
of an individual prediction may be as important as its estimated probability [24–29].Where the paper says it
Where the paper says it
Limitations
Several limitations should be considered, including the reliance on pretreatment CT and radiation dose distributions as primary model inputs. The predominance of aleatoric uncertainty observed in the analysis suggests the need for future studies incorporating toxicity-rich longitudinal data acquired during and after RT. p. 3 Clinical
utility is further evaluated using decision curve analysis to determine whether incorporating uncertainty-aware RP risk estimates could provide greater net benefit across
clinically relevant decision thresholds. By combining individualized RP risk prediction
with explicit characterization of aleatoric, epistemic, and label uncertainty, MM-DiT
provides not only an estimate of RP risk but also information regarding the reliability
of that estimate. Clinical
utility is further evaluated using decision curve analysis to determine whether incorporating uncertainty-aware RP risk estimates could provide greater net benefit across
clinically relevant decision thresholds. By combining individualized RP risk prediction
with explicit characterization of aleatoric, epistemic, and label uncertainty, MM-DiT
provides not only an estimate of RP risk but also information regarding the reliability
of that estimate. We evaluate MM-DiT in two independent cohorts using complementary assessments of predictive discrimination, calibration, and uncertainty quantification. Clinical
utility is further evaluated using decision curve analysis to determine whether incorporating uncertainty-aware RP risk estimates could provide greater net benefit across
clinically relevant decision thresholds.Where the paper says it
Where the paper says it
Where the paper says it
Conclusion
In this study, MM-DiT developed a Bayesian multimodal framework that integrates planning CT imaging and 3D radiation dose distributions for individualized RP risk prediction and explicit uncertainty quantification. The framework extends conventional RP risk stratification toward uncertainty-aware clinical interpretation. p. 3 By combining individualized RP risk prediction
with explicit characterization of aleatoric, epistemic, and label uncertainty, MM-DiT
provides not only an estimate of RP risk but also information regarding the reliability
of that estimate. This uncertainty-aware framework may provide a more informative basis for risk stratification and clinical decision-making, supporting the potential
translation of AI-assisted RP prediction into routine thoracic radiation oncology.Where the paper says it
Acknowledgment
This manuscript was prepared using data from Datasets from the NCTNNCORP Data Archive of the National Cancer Institute’s (NCI’s) National Clinical Trials Network (NCTN). This work was partially supported by the Radiation Oncology Institute (Grant 2022-9132). p. 3 Each patient record
includes a planning CT image set and corresponding 3D spatial dose distribution. The
first cohort is derived from the NRG Radiation Therapy Oncology Group (RTOG)
0617 randomized clinical trial and accessed through The Cancer Imaging Archive [34]
with approval from the National Cancer Institute (NCI) National Clinical Trials Network (NCTN)/National Community Oncology Research Program (NCORP).Where the paper says it
References
[1] Rahi, M. S. et al. Radiation-induced lung injury—current perspectives and management. p. 1 Accurate identification of patients at high risk for RP before treatment could facilitate early risk stratification and more informed clinical management, providing a
foundation for individualized treatment decisions and improved patient outcomes. Current clinical practice for reducing RP risk relies on lung dose-volume histograms
(DVHs), which summarize the relationship between radiation dose and the volume
of healthy lung receiving that dose.Where the paper says it
[2] Kaesmann, L. et al. Radiation-induced lung toxicity–cellular and molecular mechanisms of pathogenesis, management, and literature review. p. 0 org;
Abstract
Radiation pneumonitis (RP) is a common and clinically significant toxicity of
thoracic radiation therapy that can cause pulmonary morbidity and impair quality of life. Although conventional dose-volume histogram–based metrics and
normal tissue complication probability models are widely used for RP risk
assessment, they inadequately capture the complex spatial, anatomical, and
patient-specific factors underlying radiation-induced lung injury.Where the paper says it
[3] Kraus, K. M. et al. Dosiomics and radiomics-based prediction of pneumonitis after radiotherapy and immune checkpoint inhibition: The relevance of fractionation. p. 0 org;
Abstract
Radiation pneumonitis (RP) is a common and clinically significant toxicity of
thoracic radiation therapy that can cause pulmonary morbidity and impair quality of life. Although conventional dose-volume histogram–based metrics and
normal tissue complication probability models are widely used for RP risk
assessment, they inadequately capture the complex spatial, anatomical, and
patient-specific factors underlying radiation-induced lung injury.Where the paper says it
[4] Weisman, M. et al. Evaluation of radiation pneumonitis in a phase 2 study of consolidation immunotherapy with nivolumab and ipilimumab or nivolumab alone following concurrent chemoradiation therapy for unresectable stage iiia/iiib nonsmall cell lung cancer. p. 3 The
first cohort is derived from the NRG Radiation Therapy Oncology Group (RTOG)
0617 randomized clinical trial and accessed through The Cancer Imaging Archive [34]
with approval from the National Cancer Institute (NCI) National Clinical Trials Network (NCTN)/National Community Oncology Research Program (NCORP). The trial
includes 544 patients with stage III non-small-cell lung cancer (NSCLC) who were
enrolled in a randomized study evaluating radiation dose escalation in combination
of chemotherapy, with and without cetuximab [35, 36].Where the paper says it
[5] Xiao, L. et al. Early prediction of radiation pneumonitis in patients with lung cancer treated with immunotherapy through monitoring of plasma chemokines. p. 1 Accurate identification of patients at high risk for RP before treatment could facilitate early risk stratification and more informed clinical management, providing a
foundation for individualized treatment decisions and improved patient outcomes. Current clinical practice for reducing RP risk relies on lung dose-volume histograms
(DVHs), which summarize the relationship between radiation dose and the volume
of healthy lung receiving that dose.Where the paper says it
[6] Lyman, J. p. 1 During treatment planning, clinically established
dose constraints derived from DVH metrics, such as mean lung dose (MLD), V5 (lung
volume receiving ≥ 5Gy), V20 (lung volume receiving ≥ 20Gy), and V30 (lung volume receiving ≥ 30Gy), are evaluated for toxicity risk assessment. Building upon these
dosimetric indices, normal tissue complication probability (NTCP) models, such as
the Lyman-Kutcher-Burman (LKB) model [6–8], have become widely used to estimate
the probability of radiation-induced lung injury [9, 10]. org;
Abstract
Radiation pneumonitis (RP) is a common and clinically significant toxicity of
thoracic radiation therapy that can cause pulmonary morbidity and impair quality of life. Although conventional dose-volume histogram–based metrics and
normal tissue complication probability models are widely used for RP risk
assessment, they inadequately capture the complex spatial, anatomical, and
patient-specific factors underlying radiation-induced lung injury.Where the paper says it
Where the paper says it
[7] Kutcher, G. p. 1 During treatment planning, clinically established
dose constraints derived from DVH metrics, such as mean lung dose (MLD), V5 (lung
volume receiving ≥ 5Gy), V20 (lung volume receiving ≥ 20Gy), and V30 (lung volume receiving ≥ 30Gy), are evaluated for toxicity risk assessment. Building upon these
dosimetric indices, normal tissue complication probability (NTCP) models, such as
the Lyman-Kutcher-Burman (LKB) model [6–8], have become widely used to estimate
the probability of radiation-induced lung injury [9, 10]. During treatment planning, clinically established
dose constraints derived from DVH metrics, such as mean lung dose (MLD), V5 (lung
volume receiving ≥ 5Gy), V20 (lung volume receiving ≥ 20Gy), and V30 (lung volume receiving ≥ 30Gy), are evaluated for toxicity risk assessment. Building upon these
dosimetric indices, normal tissue complication probability (NTCP) models, such as
the Lyman-Kutcher-Burman (LKB) model [6–8], have become widely used to estimate
the probability of radiation-induced lung injury [9, 10]. org;
Abstract
Radiation pneumonitis (RP) is a common and clinically significant toxicity of
thoracic radiation therapy that can cause pulmonary morbidity and impair quality of life. Although conventional dose-volume histogram–based metrics and
normal tissue complication probability models are widely used for RP risk
assessment, they inadequately capture the complex spatial, anatomical, and
patient-specific factors underlying radiation-induced lung injury.Where the paper says it
Where the paper says it
Where the paper says it
[8] Kutcher, G. p. 1 During treatment planning, clinically established
dose constraints derived from DVH metrics, such as mean lung dose (MLD), V5 (lung
volume receiving ≥ 5Gy), V20 (lung volume receiving ≥ 20Gy), and V30 (lung volume receiving ≥ 30Gy), are evaluated for toxicity risk assessment. Building upon these
dosimetric indices, normal tissue complication probability (NTCP) models, such as
the Lyman-Kutcher-Burman (LKB) model [6–8], have become widely used to estimate
the probability of radiation-induced lung injury [9, 10]. During treatment planning, clinically established
dose constraints derived from DVH metrics, such as mean lung dose (MLD), V5 (lung
volume receiving ≥ 5Gy), V20 (lung volume receiving ≥ 20Gy), and V30 (lung volume receiving ≥ 30Gy), are evaluated for toxicity risk assessment. Building upon these
dosimetric indices, normal tissue complication probability (NTCP) models, such as
the Lyman-Kutcher-Burman (LKB) model [6–8], have become widely used to estimate
the probability of radiation-induced lung injury [9, 10]. During treatment planning, clinically established
dose constraints derived from DVH metrics, such as mean lung dose (MLD), V5 (lung
volume receiving ≥ 5Gy), V20 (lung volume receiving ≥ 20Gy), and V30 (lung volume receiving ≥ 30Gy), are evaluated for toxicity risk assessment. Building upon these
dosimetric indices, normal tissue complication probability (NTCP) models, such as
the Lyman-Kutcher-Burman (LKB) model [6–8], have become widely used to estimate
the probability of radiation-induced lung injury [9, 10].Where the paper says it
Where the paper says it
Where the paper says it
[9] He, Y. et al. Quantifying the effect of 4-dimensional computed tomography–based deformable dose accumulation on representing radiation damage for patients with locally advanced non-small cell lung cancer treated with standard-fractionated intensity-modulated radiation therapy. p. 3 The
first cohort is derived from the NRG Radiation Therapy Oncology Group (RTOG)
0617 randomized clinical trial and accessed through The Cancer Imaging Archive [34]
with approval from the National Cancer Institute (NCI) National Clinical Trials Network (NCTN)/National Community Oncology Research Program (NCORP). The trial
includes 544 patients with stage III non-small-cell lung cancer (NSCLC) who were
enrolled in a randomized study evaluating radiation dose escalation in combination
of chemotherapy, with and without cetuximab [35, 36].Where the paper says it
[10] Liu, Y. p. 0 Multimodal Deep Learning for Uncertainty-Aware
Radiation Pneumonitis Risk Prediction
Jin Yang, Tian Liu, Jing Wang, Robert Samstein,
Kenneth Rosenzweig, Julie Bloom, Ming Chao*
Department of Radiation Oncology, Icahn School of Medicine at Mount
Sinai, New York, NY 10029, USA. *Corresponding author(s). During treatment planning, clinically established
dose constraints derived from DVH metrics, such as mean lung dose (MLD), V5 (lung
volume receiving ≥ 5Gy), V20 (lung volume receiving ≥ 20Gy), and V30 (lung volume receiving ≥ 30Gy), are evaluated for toxicity risk assessment. Building upon these
dosimetric indices, normal tissue complication probability (NTCP) models, such as
the Lyman-Kutcher-Burman (LKB) model [6–8], have become widely used to estimate
the probability of radiation-induced lung injury [9, 10]. The
first cohort is derived from the NRG Radiation Therapy Oncology Group (RTOG)
0617 randomized clinical trial and accessed through The Cancer Imaging Archive [34]
with approval from the National Cancer Institute (NCI) National Clinical Trials Network (NCTN)/National Community Oncology Research Program (NCORP). The trial
includes 544 patients with stage III non-small-cell lung cancer (NSCLC) who were
enrolled in a randomized study evaluating radiation dose escalation in combination
of chemotherapy, with and without cetuximab [35, 36].Where the paper says it
Where the paper says it
Where the paper says it
[11] Marks, L. B. et al. Radiation dose–volume effects in the lung. p. 0 org;
Abstract
Radiation pneumonitis (RP) is a common and clinically significant toxicity of
thoracic radiation therapy that can cause pulmonary morbidity and impair quality of life. Although conventional dose-volume histogram–based metrics and
normal tissue complication probability models are widely used for RP risk
assessment, they inadequately capture the complex spatial, anatomical, and
patient-specific factors underlying radiation-induced lung injury.Where the paper says it
[12] Van den Bosch, L. et al. Key challenges in normal tissue complication probability model development and validation: towards a comprehensive strategy. p. 1 During treatment planning, clinically established
dose constraints derived from DVH metrics, such as mean lung dose (MLD), V5 (lung
volume receiving ≥ 5Gy), V20 (lung volume receiving ≥ 20Gy), and V30 (lung volume receiving ≥ 30Gy), are evaluated for toxicity risk assessment. Building upon these
dosimetric indices, normal tissue complication probability (NTCP) models, such as
the Lyman-Kutcher-Burman (LKB) model [6–8], have become widely used to estimate
the probability of radiation-induced lung injury [9, 10].Where the paper says it
[13] Valdes, G., Solberg, T. D., Heskel, M., Ungar, L. & Simone, C. B. Using machine learning to predict radiation pneumonitis in patients with stage i non-small cell lung cancer treated with stereotactic body radiation therapy. p. 3 The
first cohort is derived from the NRG Radiation Therapy Oncology Group (RTOG)
0617 randomized clinical trial and accessed through The Cancer Imaging Archive [34]
with approval from the National Cancer Institute (NCI) National Clinical Trials Network (NCTN)/National Community Oncology Research Program (NCORP). The trial
includes 544 patients with stage III non-small-cell lung cancer (NSCLC) who were
enrolled in a randomized study evaluating radiation dose escalation in combination
of chemotherapy, with and without cetuximab [35, 36].Where the paper says it
[14] Krafft, S. P. et al. The utility of quantitative ct radiomics features for improved prediction of radiation pneumonitis. p. 0 Although conventional dose-volume histogram–based metrics and
normal tissue complication probability models are widely used for RP risk
assessment, they inadequately capture the complex spatial, anatomical, and
patient-specific factors underlying radiation-induced lung injury. Recent machine
learning approaches have improved RP risk prediction by integrating multimodal
clinical and imaging information; however, most provide a point risk estimate
without quantifying the reliability of individual predictions, limiting their potential clinical utility.Where the paper says it
[15] Yu, H. et al. Machine learning to build and validate a model for radiation pneumonitis prediction in patients with non small cell lung cancer. p. 3 The
first cohort is derived from the NRG Radiation Therapy Oncology Group (RTOG)
0617 randomized clinical trial and accessed through The Cancer Imaging Archive [34]
with approval from the National Cancer Institute (NCI) National Clinical Trials Network (NCTN)/National Community Oncology Research Program (NCORP). The trial
includes 544 patients with stage III non-small-cell lung cancer (NSCLC) who were
enrolled in a randomized study evaluating radiation dose escalation in combination
of chemotherapy, with and without cetuximab [35, 36].Where the paper says it
[16] Luna, J. M. et al. Predicting radiation pneumonitis in locally advanced stage ii–iii non-small cell lung cancer using machine learning. p. 3 The
first cohort is derived from the NRG Radiation Therapy Oncology Group (RTOG)
0617 randomized clinical trial and accessed through The Cancer Imaging Archive [34]
with approval from the National Cancer Institute (NCI) National Clinical Trials Network (NCTN)/National Community Oncology Research Program (NCORP). The trial
includes 544 patients with stage III non-small-cell lung cancer (NSCLC) who were
enrolled in a randomized study evaluating radiation dose escalation in combination
of chemotherapy, with and without cetuximab [35, 36].Where the paper says it
[17] Hirose, T.-a. et al. Radiomic prediction of radiation pneumonitis on pretreatment planning computed tomography images prior to lung cancer stereotactic body radiation therapy. p. 1 This framework may ultimately support more
informed individualized treatment planning and safer clinical decision-making in
thoracic radiation therapy. Keywords: Radiation Pneumonitis, Self-supervised Multimodal Learning, Bayesian
Approximation, Predictive Uncertainty, Thoracic Radiation Therapy
1 Introduction
Radiation pneumonitis (RP) is a common and clinically significant toxicity of thoracic radiation therapy (RT) for lung cancer and other thoracic malignancies and
represents an important constraint on the safe delivery of radiation dose [1, 2].Where the paper says it
[18] Feng, A. et al. Improvement of prediction performance for radiation pneumonitis by using 3-dimensional dosiomic features. p. 5 CT
Dose distribution
Multi-modal features 𝐹
Bayesian classifier
True Predictive
distribution
Encoder Encoder
Latent space
PoE/MoE
CT
𝒩(𝜇1, 𝜎1
2𝑰)
𝒩(𝜇2, 𝜎2
2𝑰)
Dose distribution
𝒩(𝜇, 𝜎
2𝑰)
𝑓𝜃1(Θ)
𝑓𝜃2(Θ)
Decoder Decoder
𝑔𝜃1(Θ)
𝑔𝜃2(Θ)
Reconstructed CT
Reconstructed Dose
FC
ReLU
MC Dropout
FC
RP Risk Prediction
Aleatoric Uncertainty
Epistemic Uncertainty Label Noise Transition Layer
False positive rate
False negative rate
Observed Predictive
distribution
Label
Uncertainty
Modality
Reconstruction
ℒ𝑅𝑒𝑐
Weighted Binary Cross
Entropy Loss ℒ𝑊𝐵𝐶𝐸
RP grade
Diffusion
Transformer (DiT) Noise
Diffusion
Loss ℒ𝐷𝑖𝑓𝑓
Timestep t
Self-supervised pre
-training Supervised Fine
-tuning
Fig. 1: Multimodal Bayesian Diffusion Transformer (MM-DiT) framework
for radiation pneumonitis risk prediction and uncertainty quantification.Where the paper says it
[19] Gadsby, A. et al. Impact of normal lung volume choices on radiation pneumonitis risk prediction in locally non-small cell lung cancer radiation therapy. p. 3 The
first cohort is derived from the NRG Radiation Therapy Oncology Group (RTOG)
0617 randomized clinical trial and accessed through The Cancer Imaging Archive [34]
with approval from the National Cancer Institute (NCI) National Clinical Trials Network (NCTN)/National Community Oncology Research Program (NCORP). The trial
includes 544 patients with stage III non-small-cell lung cancer (NSCLC) who were
enrolled in a randomized study evaluating radiation dose escalation in combination
of chemotherapy, with and without cetuximab [35, 36].Where the paper says it
[20] Cui, S., Ten Haken, R. K. & El Naqa, I. Integrating multiomics information in deep learning architectures for joint actuarial outcome prediction in non-small cell lung cancer patients after radiation therapy. p. 3 The
first cohort is derived from the NRG Radiation Therapy Oncology Group (RTOG)
0617 randomized clinical trial and accessed through The Cancer Imaging Archive [34]
with approval from the National Cancer Institute (NCI) National Clinical Trials Network (NCTN)/National Community Oncology Research Program (NCORP). The trial
includes 544 patients with stage III non-small-cell lung cancer (NSCLC) who were
enrolled in a randomized study evaluating radiation dose escalation in combination
of chemotherapy, with and without cetuximab [35, 36].Where the paper says it
[21] Jiang, W., Song, Y., Sun, Z., Qiu, J. & Shi, L. Dosimetric factors and radiomics features within different regions of interest in planning ct images for improving the prediction of radiation pneumonitis. p. 1 Furthermore, RP is a multifactorial process influenced not only by radiation
dose but also by patient-specific characteristics, pulmonary function, tumor location,
anatomical imaging features, and individual radiosensitivity [11, 12]. Consequently,
patients with nearly identical DVH metrics can experience markedly different clinical
outcomes, underscoring the limited ability of traditional dosimetric models to capture
the complex biological and patient-specific factors underlying radiation-induced lung
injury.Where the paper says it
[22] Jiao, Y. et al. Integrating radiomics and dosiomics with lung biologically equivalent dose for predicting symptomatic radiation pneumonitis after lung sbrt: A dual-center study. p. 0 Although conventional dose-volume histogram–based metrics and
normal tissue complication probability models are widely used for RP risk
assessment, they inadequately capture the complex spatial, anatomical, and
patient-specific factors underlying radiation-induced lung injury. Recent machine
learning approaches have improved RP risk prediction by integrating multimodal
clinical and imaging information; however, most provide a point risk estimate
without quantifying the reliability of individual predictions, limiting their potential clinical utility.Where the paper says it
[23] Reuter, L. M. et al. Prediction of symptomatic radiation pneumonitis in lung cancer patients: A radiomics and dosiomics machine learning approach using the prospective multicenter rtog 0617 and requite trials. p. 3 The
first cohort is derived from the NRG Radiation Therapy Oncology Group (RTOG)
0617 randomized clinical trial and accessed through The Cancer Imaging Archive [34]
with approval from the National Cancer Institute (NCI) National Clinical Trials Network (NCTN)/National Community Oncology Research Program (NCORP). The trial
includes 544 patients with stage III non-small-cell lung cancer (NSCLC) who were
enrolled in a randomized study evaluating radiation dose escalation in combination
of chemotherapy, with and without cetuximab [35, 36].Where the paper says it
[24] Ovadia, Y. et al. Can you trust your model’s uncertainty? p. 0 The resulting representations are further refined using
a latent diffusion transformer and transferred to a Bayesian prediction framework
for probabilistic RP risk estimation. A learnable label-noise model is incorporated
to explicitly account for uncertainty arising from imperfect toxicity annotations. The
first cohort is derived from the NRG Radiation Therapy Oncology Group (RTOG)
0617 randomized clinical trial and accessed through The Cancer Imaging Archive [34]
with approval from the National Cancer Institute (NCI) National Clinical Trials Network (NCTN)/National Community Oncology Research Program (NCORP). The trial
includes 544 patients with stage III non-small-cell lung cancer (NSCLC) who were
enrolled in a randomized study evaluating radiation dose escalation in combination
of chemotherapy, with and without cetuximab [35, 36].Where the paper says it
Where the paper says it
[25] Loftus, T. J. et al. Uncertainty-aware deep learning in healthcare: a scoping review. p. 0 Multimodal Deep Learning for Uncertainty-Aware
Radiation Pneumonitis Risk Prediction
Jin Yang, Tian Liu, Jing Wang, Robert Samstein,
Kenneth Rosenzweig, Julie Bloom, Ming Chao*
Department of Radiation Oncology, Icahn School of Medicine at Mount
Sinai, New York, NY 10029, USA. *Corresponding author(s).Where the paper says it
[26] Yang, S. p. 0 Multimodal Deep Learning for Uncertainty-Aware
Radiation Pneumonitis Risk Prediction
Jin Yang, Tian Liu, Jing Wang, Robert Samstein,
Kenneth Rosenzweig, Julie Bloom, Ming Chao*
Department of Radiation Oncology, Icahn School of Medicine at Mount
Sinai, New York, NY 10029, USA. *Corresponding author(s). The resulting representations are further refined using
a latent diffusion transformer and transferred to a Bayesian prediction framework
for probabilistic RP risk estimation. A learnable label-noise model is incorporated
to explicitly account for uncertainty arising from imperfect toxicity annotations.Where the paper says it
Where the paper says it
[27] Wei, Y. et al. Deep learning with noisy labels in medical prediction problems: a scoping review. p. 5 MM-DiT is a multimodal deep learning framework that performs RP
prediction through two sequential stages: self-supervised multimodal representation
learning and supervised Bayesian fine-tuning. During self-supervised pre-training, a
multi-modal Diffusion Transformer (DiT) learns cross-modal representations from
planning CT images and 3D dose distributions without using RP labels.Where the paper says it
[28] Tyralis, H. & Papacharalampous, G. A review of predictive uncertainty estimation with machine learning. p. 0 Recent machine
learning approaches have improved RP risk prediction by integrating multimodal
clinical and imaging information; however, most provide a point risk estimate
without quantifying the reliability of individual predictions, limiting their potential clinical utility. We propose a Multimodal Bayesian Diffusion Transformer
(MM-DiT) framework that jointly estimates RP risk and characterizes the
sources of predictive uncertainty.Where the paper says it
[29] Stutz, D. et al. Evaluating medical ai systems in dermatology under uncertain ground truth. p. 5 A Diffusion Transformer (DiT) is subsequently employed to
learn robust multimodal representations from the fused latent distributions. During
self-supervised pre-training, the encoders and DiT are optimized without access to
ground truth RP labels, enabling the model to learn disease-agnostic anatomical and
6Where the paper says it
[30] Kurz, A. et al. Uncertainty estimation in medical image classification: systematic review. p. 2 Despite the growing recognition of uncertainty as an important component of
trustworthy medical AI [30, 31], uncertainty-aware approaches remain largely unexplored in RP prediction [32, 33]. RP risk estimation is inherently subject to multiple
sources of uncertainty, including variability in patient-specific susceptibility and the
complex relationship between radiation exposure and lung injury, limited information
available to the predictive model, ambiguity or errors in clinical toxicity assessment.Where the paper says it
[31] Huang, L., Ruan, S., Xing, Y. & Feng, M. A review of uncertainty quantification in medical image analysis: Probabilistic and non-probabilistic methods. p. 3 calibrated probabilistic RP risk estimates together with complementary measures of
aleatoric, epistemic, and label uncertainty, enabling assessment of prediction reliability
and identification of cases that may warrant additional clinical evaluation. We evaluate MM-DiT in two independent cohorts using complementary assessments of predictive discrimination, calibration, and uncertainty quantification.Where the paper says it
[32] Wahid, K. A. et al. Artificial intelligence uncertainty quantification in radiotherapy applications- a scoping review. p. 2 To overcome these limitations, recent studies have increasingly explored artificial intelligence (AI), particularly machine learning (ML) and deep learning (DL),
for RP risk prediction [13–19]. Unlike conventional NTCP models, these approaches
can integrate heterogeneous sources of information, including patient demographics,
clinical characteristics, RT dose distributions, and radiomic features extracted from
imaging such as computed tomography (CT).Where the paper says it
[33] D¨unger, L., M¨ausel, E., Zwanenburg, A. & L¨ock, S. Uncertainties in outcome modelling in radiation oncology. p. 2 MM-DiT integrates planning CT images and 3D
radiation dose distributions to estimate both an individual patient’s probability of
developing RP and the uncertainty associated with that prediction. Rather than
relying solely on supervised learning from limited toxicity labels, MM-DiT first
learns multimodal representations through self-supervised pre-training using routinely
acquired planning data, reducing dependence on potentially noisy outcome labels
and promoting robust representation learning.Where the paper says it
Pruning redundant reasoning tokens
The authors propose MM-DiT to provide individualized RP probabilities together with complementary measures of aleatoric, epistemic, and label uncertainty. This framework distinguishes between what it predicts and how certain that prediction is. p. 0 Recent machine
learning approaches have improved RP risk prediction by integrating multimodal
clinical and imaging information; however, most provide a point risk estimate
without quantifying the reliability of individual predictions, limiting their potential clinical utility. We propose a Multimodal Bayesian Diffusion Transformer
(MM-DiT) framework that jointly estimates RP risk and characterizes the
sources of predictive uncertainty. calibrated probabilistic RP risk estimates together with complementary measures of
aleatoric, epistemic, and label uncertainty, enabling assessment of prediction reliability
and identification of cases that may warrant additional clinical evaluation. We evaluate MM-DiT in two independent cohorts using complementary assessments of predictive discrimination, calibration, and uncertainty quantification.Where the paper says it
Where the paper says it
How the training signal is built
The model first learns multimodal representations through self-supervised pre-training using routinely acquired planning data, reducing dependence on potentially noisy outcome labels. This stage utilizes a multi-modal Diffusion Transformer (DiT) to learn cross-modal representations from planning CT images and 3D dose distributions without using RP labels. p. 5 MM-DiT is a multimodal deep learning framework that performs RP
prediction through two sequential stages: self-supervised multimodal representation
learning and supervised Bayesian fine-tuning. During self-supervised pre-training, a
multi-modal Diffusion Transformer (DiT) learns cross-modal representations from
planning CT images and 3D dose distributions without using RP labels. MM-DiT integrates planning CT images and 3D
radiation dose distributions to estimate both an individual patient’s probability of
developing RP and the uncertainty associated with that prediction. Rather than
relying solely on supervised learning from limited toxicity labels, MM-DiT first
learns multimodal representations through self-supervised pre-training using routinely
acquired planning data, reducing dependence on potentially noisy outcome labels
and promoting robust representation learning. CT
Dose distribution
Multi-modal features 𝐹
Bayesian classifier
True Predictive
distribution
Encoder Encoder
Latent space
PoE/MoE
CT
𝒩(𝜇1, 𝜎1
2𝑰)
𝒩(𝜇2, 𝜎2
2𝑰)
Dose distribution
𝒩(𝜇, 𝜎
2𝑰)
𝑓𝜃1(Θ)
𝑓𝜃2(Θ)
Decoder Decoder
𝑔𝜃1(Θ)
𝑔𝜃2(Θ)
Reconstructed CT
Reconstructed Dose
FC
ReLU
MC Dropout
FC
RP Risk Prediction
Aleatoric Uncertainty
Epistemic Uncertainty Label Noise Transition Layer
False positive rate
False negative rate
Observed Predictive
distribution
Label
Uncertainty
Modality
Reconstruction
ℒ𝑅𝑒𝑐
Weighted Binary Cross
Entropy Loss ℒ𝑊𝐵𝐶𝐸
RP grade
Diffusion
Transformer (DiT) Noise
Diffusion
Loss ℒ𝐷𝑖𝑓𝑓
Timestep t
Self-supervised pre
-training Supervised Fine
-tuning
Fig. 1: Multimodal Bayesian Diffusion Transformer (MM-DiT) framework
for radiation pneumonitis risk prediction and uncertainty quantification.Where the paper says it
Where the paper says it
Where the paper says it
Where the gains come from
The CNN+PoE configuration achieved the highest overall performance across both cohorts, with AUC values reaching 85.59% in the RTOG 0617 cohort and 89.11% in the MSHS cohort. The superior discriminative ability of MM-DiT is demonstrated by its higher AUC compared to single-modality models and other multimodal VAEs. The framework supports risk-based decision-making across a range of clinically relevant thresholds, showing a positive net benefit in decision curve analysis. p. 3 We evaluate MM-DiT in two independent cohorts using complementary assessments of predictive discrimination, calibration, and uncertainty quantification. Clinical
utility is further evaluated using decision curve analysis to determine whether incorporating uncertainty-aware RP risk estimates could provide greater net benefit across
clinically relevant decision thresholds.Where the paper says it
The joint consideration of predicted risk and predictive uncertainty provides a framework for clinically interpreting model outputs. This leads to four risk–uncertainty groups, such as High Risk + High Confidence, which suggests immediate clinical intervention. p. 1 Preventing RP while maintaining optimal treatment outcomes therefore remains a central challenge in thoracic RT. Accurate identification of patients at high risk for RP before treatment could facilitate early risk stratification and more informed clinical management, providing a
foundation for individualized treatment decisions and improved patient outcomes. By combining individualized RP risk prediction
with explicit characterization of aleatoric, epistemic, and label uncertainty, MM-DiT
provides not only an estimate of RP risk but also information regarding the reliability
of that estimate. This uncertainty-aware framework may provide a more informative basis for risk stratification and clinical decision-making, supporting the potential
translation of AI-assisted RP prediction into routine thoracic radiation oncology.Where the paper says it
Where the paper says it
The present study was retrospective and relied on pretreatment CT and radiation dose distributions as the primary model inputs. The predominance of aleatoric uncertainty observed in the analysis suggests the need for future studies incorporating toxicity-rich longitudinal data acquired during and after RT. Furthermore, additional multi-institutional validation across diverse patient populations and imaging protocols is needed to establish generalizability. p. 1 First, DVHs compress complex threedimensional (3D) dose distributions into one-dimensional summary statistics, thereby
discarding spatial information regarding regional dose deposition and tissue heterogeneity. Furthermore, RP is a multifactorial process influenced not only by radiation
dose but also by patient-specific characteristics, pulmonary function, tumor location,
anatomical imaging features, and individual radiosensitivity [11, 12]. Consequently, even when a model demonstrates good overall discrimination, its output for
a specific patient may not indicate how strongly that prediction is supported by the
available evidence. This limitation is particularly important when RP risk estimates
are used to inform treatment planning or clinical management, where the reliability
of an individual prediction may be as important as its estimated probability [24–29].Where the paper says it
Where the paper says it
In this study, MM-DiT developed a Bayesian multimodal framework that integrates planning CT imaging and 3D radiation dose distributions for individualized RP risk prediction and explicit uncertainty quantification. This framework provides a more transparent and reliable approach to AI-assisted toxicity assessment. p. 1 representation learning, Bayesian uncertainty quantification, and explicit modeling of label noise, MM-DiT offers a more reliable and transparent approach to
AI-assisted RP risk assessment. This framework may ultimately support more
informed individualized treatment planning and safer clinical decision-making in
thoracic radiation therapy.Where the paper says it
Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.
Improvements for AI systems
-
Multimodal representation learning via self-supervised pre-training using a Diffusion Transformer (DiT) learns disease agnostic anatomical and dosimetric representations without relying on potentially noisy toxicity labels, thereby reducing dependence on limited outcome labels.
-
Bayesian prediction framework with Monte Carlo Dropout approximates the posterior distribution over model parameters, generating a distribution of RP risk predictions rather than a single point estimate for each patient.
-
Explicit modeling of uncertainty decomposes total predictive uncertainty into aleatoric, epistemic, and label uncertainty, allowing clinicians to distinguish between inherent data noise, model limitations due to insufficient training data (epistemic), and inconsistencies in toxicity annotations (label).
-
The framework provides a clinical interpretation framework by stratifying patients into four risk-uncertainty groups—Low Risk High Confidence, Low Risk Low Confidence, High Risk High Confidence, and High Risk Low Confidence—to guide decisions like standard treatment or immediate intervention.
Abstract
Radiation pneumonitis (RP) is a potentially serious complication of thoracic radiotherapy, making reliable risk prediction important for identifying high-risk patients and guiding effective toxicity management. Although deep learning models can predict RP risk, uncertainty quantification remains underdeveloped in radiation toxicity assessment. We developed a Multimodal Bayesian Diffusion Transformer (MM-DiT) that integrates pretreatment computed tomography (CT) and three-dimensional planning dose distributions for RP risk prediction and uncertainty quantification. MM-DiT combines self-supervised multimodal representation learning with Bayesian classification to estimate individual risk and characterize aleatoric, epistemic, and label uncertainty. In independent Mount Sinai Health System (MSHS) and RTOG cohorts, MM-DiT achieved areas under the receiver operating characteristic curve (AUCs) of 0.89 and 0.86, respectively, with expected calibration errors of 0.02 and 0.03. Incorporating post-treatment CT increased AUC to 0.95 in MSHS and reduced aleatoric and epistemic uncertainty by approximately 40-45% and 40%, respectively. These findings establish a framework for uncertainty-aware radiation toxicity prediction that complements risk estimates with information about predictive reliability. By integrating multimodal learning and uncertainty quantification, MM-DiT may advance more interpretable and trustworthy AI-assisted toxicity assessment and support individualized radiotherapy planning.
Related papers
- HDNet: High-resolution Dual-domain Learning for Spectral Compressive Imaging
- Mask-guided Spectral-wise Transformer for Efficient Hyperspectral Image Reconstruction
- FDIR: Fidelity-Anchored Decompressed Image Restoration for Human and Machine Perception
- A Disease-Centric Vision-Language Foundation Model for Precision Oncology in Kidney Cancer
- CADS: A Comprehensive Anatomical Dataset and Segmentation for Whole-Body Anatomy in Computed Tomography
- Task-Aware Bitrate Thresholds for Codec-Robust Ultrasound Segmentation