From Vision to Context: A Review of Prompt-Driven Target Volume Segmentation in Radiotherapy
Listen
Radio episode about this paper
Transcript
Introduction to the show: ident: Electrical Engineering. The day in electrical engineering: signals, systems and the hardware that carries them.
Rosa: We're looking at the paper: "From Vision to Context".
Dev: Here's the gist of the paper. Accurate target volume (TV) segmentation is essential for radiotherapy (RT) planning but remains time-consuming and dependent on expert knowledge This work aims to review these approaches for RT TV segmentation with four objectives.
Rosa: Manual TV delineation remains one of the most time-consuming and error-prone steps in the RT workflow This error-proneness is most evident in interobserver variability (IOV) A large body of evidence has shown that IOV in TV contouring is a major source of geometric uncertainty in RT planning.
Dev: Substantial IOV has been reported across multiple cancer sites, including lung, prostate, and esophageal cancer.
Rosa: IOV arises from multiple factors, including radiation oncologist expertise, image quality, and the inherent subjectivity in interpreting ambiguous tumor boundaries on medical images.
Paper summary: Rosa: So we're looking at this paper, "From Vision to Context: A Review of Prompt-Driven Target Volume Segmentation in Radiotherapy." Basically, the main point is that accurate target volume segmentation is super important for radiotherapy planning but it still takes too much time and depends on the radiation oncologist's expertise.
Dev: Yeah, and image-only models just can't capture that clinical reasoning needed for the clinical target volume or CTV, where the boundaries are defined by things a doctor knows rather than just what’s visible in a picture.
Rosa: Exactly. And this paper reviews different ways people are trying to fix that limitation using prompt-driven segmentation as a strategy to bring in that missing context.
Taro: So what's the big question here? What exactly is this review comparing across all these prompt methods?
Dev: The authors set four objectives for this review. They want to compare the different prompting techniques being used, see how they actually perform against those image-only baselines, look at how they are evaluated using standard metrics like Dice similarity coefficient and 95th Hausdorff distance, sensitivity and precision, and finally figure out what gaps we need to fill for actual use in a clinic <ref:2611.06570#pg1>.
Rosa: It’s interesting because the search strategy they used was pretty thorough. They looked across PubMed, Embase, and arXiv for studies that used geometric prompts, clinical text inputs, or interactive prompting into deep learning models for radiotherapy target volume delineation.
Dev: That search pulled in thirteen studies overall that covered a range of cancers like head and neck, breast, prostate cancer, lung cancer and rectal cancer. Out of those thirteen studies they found seven that used clinical text prompting four that used geometric prompting and two that used interactive prompting, plus three studies combined both modalities.
Rosa: The performance numbers they reported are pretty encouraging but there are some caveats. They mentioned reported Dice similarity coefficients ranged from zero point sixty-seven to zero point ninety-three which is compared against the image-only baselines scoring from zero point sixty-five to zero point ninety-one.
Dev: But then they also noted that none of the studies reported sensitivity or precision metrics for these segmentation models. They focused mostly on that Dice score and the Hausdorff distance measures like HD95.
Taro: So if we only look at those numbers, it seems like prompt-driven models are generally better than image-only ones, but what does that actually mean for a patient getting treatment?
Rosa: The paper points out that clinical text prompting seems to be particularly well suited for delineating the CTV. They also noted that geometric prompting achieved a mean Dice score of zero point eighty-four on one set of test patients from four institutions, which was better than five image-only models.
Dev: Clinical text prompting hit a Dice score of zero point eighty-three on three hundred seven internal breast cancer patients and kept similar performance on two other external groups at scores of zero point eighty-two and zero point eighty-four respectively. That suggests a lot of value from the clinical context being fed into the model.
Rosa: The paper is careful about what it says, though. It highlights that the Dice score itself doesn't tell you if the contouring is too big or too small; a certain Dice value could mean under-contouring or over-contouring, or both happening at once.
Dev: And they explicitly state that no metric was reported for sensitivity and precision, which is a major gap when we think about clinical safety and how much error we can tolerate.
Taro: So what’s the next logical step according to the authors? What do they suggest is needed before this technology can actually move into routine radiotherapy practice?
Rosa: They say that geometric scores alone might not tell you if a clinician would actually accept the predicted contour. So, it seems like the next step is figuring out if these geometric improvements translate into actual dosimetric improvements for the patient.
Dev: And they point to generalizability across different radiotherapy centers as the next big thing for clinical translation because that’s where things often get messy in real practice.
Taro: So, we see models doing better than just looking at pictures, but we still have to figure out how to make sure those geometric improvements actually result in a better physical dose delivered to the tumor.
Rosa: That’s the core of it. The paper concludes that moving this from research to practice will require prospective, multiinstitutional validation and a systematic study on how clinical information should be selected, represented, and used in these models.
Dev: So for anyone listening who cares about how this changes the workflow, it means we need more than just better segmentation scores; we need real-world testing showing it works reliably across different hospitals.
Conclusion: Rosa: So we're wrapping up this review of "From Vision to Context: A Review of Prompt-Driven Target Volume Segmentation in Radiotherapy."
Dev: That paper looks at how we can get better target volume segmentation when we use deep learning, focusing on adding clinical context through prompting.
Rosa: Right, it looks at the different ways people are asking the model for extra information—whether that's geometry, text from patient reports, or interactive refinement.
Dev: The authors found that these prompt-driven models generally do better than just using image data alone for tasks like delineating the clinical target volume.
Rosa: They looked at thirteen studies across many cancer types and saw that clinical text prompting seems particularly useful for defining the actual CTV boundaries, which is where doctor knowledge matters most.
Dev: The results show Dice similarity coefficients in that range of zero point sixty-seven to zero point ninety-three when compared against image-only models.
Rosa: But here's the thing: they also pointed out that this Dice score doesn't tell you if the contour is over or under delineated; it could be both at once.
Dev: And they didn't report sensitivity or precision metrics, which means we don't know how often the model actually gets it right or what kind of error we’re dealing with.
Rosa: So the big question left hanging in that review is whether these geometric improvements translate into actual physical dose improvements for the patient.
Dev: That’s where things get tricky; moving this from a research paper to routine practice requires prospective testing across different hospitals.
Bakpen Kombata, Guus Grimbergena, Maarten L. Terpstrab, Gert J. Meijera, Lois A. Daamena
Department of Radiation Oncology, University Medical Center Utrecht · Computational Imaging Group for MR Diagnostics and Therapy, Center for Image Sciences, University Medical Center Utrecht
eess.IV
Submitted: 2026-11-04
Updated: 2026-11-04
Comments: Submitted to Computerized Medical Imaging and Graphics
License: http://creativecommons.org/licenses/by/4.0/
Importance score: 81/100
The gist: Accurate target volume (TV) segmentation is essential for radiotherapy (RT) planning but remains time-consuming and dependent on expert knowledge This work aims to review these approaches for RT TV…
Key concepts
- Target Volume (TV) Segmentation
- This is the process of accurately outlining the area within a patient's body that needs to receive radiation. Accurate segmentation is crucial for planning radiotherapy, but manual contouring is time-consuming and varies between different doctors, leading to geometric uncertainty in treatment plans.
- Prompt-Driven Models (PDMs)
- These are deep learning models enhanced by providing external information—a 'prompt'—beyond the medical image itself. This prompt can be geometric data, clinical text like patient reports, or interactive clinician input. PDMs aim to incorporate this crucial clinical reasoning into the segmentation process.
- Clinical Text Prompting
- This technique involves feeding structured or unstructured clinical information, such as TNM staging or contouring guidelines, directly into the segmentation model alongside the image data. The review found this modality particularly effective for defining Clinical Target Volumes (CTV).
- Dice Similarity Coefficient (DSC)
- DSC is a standard metric used to measure how well a predicted segmentation mask matches the ground truth (the expert contour). A higher DSC value indicates better accuracy in the segmentation, ranging from 0.67 to 0.93 in this review.
Terminology
Summary
Accurate target volume (TV) segmentation is essential for radiotherapy (RT) planning but remains time-consuming and dependent on expert knowledge This work aims to review these approaches for RT TV segmentation with four objectives. p. 0 Meijera and Lois
A. Daamena
aDepartment of Radiation Oncology, University Medical Center Utrecht, Utrecht, The Netherlands
bComputational Imaging Group for MR Diagnostics and Therapy, Center for Image Sciences, University Medical Center
Utrecht, Utrecht, The Netherlands
A R T I C L E I N F O
Keywords:
Radiotherapy
target volume
deep learning
prompt-driven segmentation
clinical text prompting
A B S T R A C T
Accurate target volume (TV) segmentation is essential for radiotherapy (RT) planning
but remains time-consuming.Where the paper says it
Introduction to the Problem
Manual TV delineation remains one of the most time-consuming and error-prone steps in the RT workflow This error-proneness is most evident in interobserver variability (IOV) A large body of evidence has shown that IOV in TV contouring is a major source of geometric uncertainty in RT planning. Substantial IOV has been reported across multiple cancer sites, including lung, prostate, and esophageal cancer. p. 0 ,
2023; Segedin and Petric, 2016). Substantial IOV has
been reported across multiple cancer sites, including
lung, prostate, and esophageal cancer (Guzene et al. , 2025; Peng
et al. IOV arises from multiple factors, including
radiation oncologist expertise, image quality, and the
inherent subjectivity in interpreting ambiguous tumor
boundaries on medical images (Guzene et al.Where the paper says it
Where the paper says it
The Need for Prompt-Driven Models
Image-only models cannot encode the clinical reasoning that governs TV delineation, particularly for the clinical target volume (CTV), where boundaries are defined by clinical knowledge rather than imaging features alone. This limitation is not expected to be resolved by developing larger architectures or training on more image data alone. p. 1 As none of these features are encoded
in the imaging, a segmentation model trained only on
image data can learn only the relationship between
image appearance and the contours provided during
training, without access to the clinical information that
determines the CTV extent. This limitation is not expected to be resolved by developing larger architectures
or training on more image data alone. This limitation is not expected to be resolved by developing larger architectures
or training on more image data alone. This challenge has motivated the development of
prompt-driven models (PDMs): segmentation models
that incorporate information that cannot be derived
from the image alongside the imaging data. This challenge has motivated the development of
prompt-driven models (PDMs): segmentation models
that incorporate information that cannot be derived
from the image alongside the imaging data. Depending
on the clinical context, this information, referred to as
a prompt, may be geometric, textual, or interactive.Where the paper says it
Where the paper says it
Where the paper says it
Classification of Prompt Modalities
Studies were classified according to the type of prompt incorporated into the segmentation process. Three prompt modalities were identified (Figure 1):
(1) geometric prompting, in which spatial inputs such as points, bounding boxes, or binary masks indicate the approximate location of the TV.
(2) clinical text prompting, in which structured clinical data such as TV location and TNM staging, or unstructured information such as clinical reports and contouring guidelines, are encoded as text and fused with imaging features. p. 1 Prompt Modality Classification
Studies were classified according to the type of
prompt incorporated into the segmentation process. Three prompt modalities were identified (Figure 1):
(1) geometric prompting, in which spatial inputs such
as points, bounding boxes, or binary masks indicate
the approximate location of the TV; (2) clinical text
prompting, in which structured clinical data such as TV
location and TNM staging, or unstructured information
such as clinical reports and contouring guidelines, are
encoded as text and fused with imaging features;
and (3) interactive prompting, in which a clinician
refines an initial model segmentation through corrective
Kombat et al.Where the paper says it
(3) interactive prompting, in which a clinician refines an initial model segmentation through corrective inputs. p. 1 Prompt Modality Classification
Studies were classified according to the type of
prompt incorporated into the segmentation process. Three prompt modalities were identified (Figure 1):
(1) geometric prompting, in which spatial inputs such
as points, bounding boxes, or binary masks indicate
the approximate location of the TV; (2) clinical text
prompting, in which structured clinical data such as TV
location and TNM staging, or unstructured information
such as clinical reports and contouring guidelines, are
encoded as text and fused with imaging features;
and (3) interactive prompting, in which a clinician
refines an initial model segmentation through corrective
Kombat et al.Where the paper says it
Performance Comparison Across Modalities
Reported DSC ranged from 0.67 to 0.93, compared to 0. p. 0 Reported
DSC ranged from 0. 93, compared to 0. Prompt-driven segmentation
has recently emerged as a strategy to address this limitation. This work aims to
review these approaches for RT TV segmentation with four objectives: (1) compare the
different prompting techniques used; (2) compare their performance against image-only
baselines; (3) assess how they are evaluated against standard metrics, including the Dice
similarity coefficient (DSC), 95th Hausdorff distance, sensitivity, precision, and clinical
assessment; and (4) identify the gaps needed for clinical translation. None reported sensitivity or precision. Clinical text prompting appears well suited to
CTV delineation. Six geometric prompting
strategies, ranging from a single center point to three
orthogonal binary masks, were simulated from the
reference GTV contours. Across 537 patients from six
institutions, nnInteractive achieved the highest median
DSC, increasing from 0. Their framework achieved
a mean DSC of 0. 84 on 120 test patients from four
institutions, outperforming five image-only models. One general-purpose segmentation
model not built specifically for RT has used this
approach, and despite being trained on a very large
and varied dataset, achieved lower accuracy than the
domain-specific models in this review (Rokuss et al. This suggests that the flexibility of free-text
prompting may come at the cost of accuracy without
training tailored to the specific clinical task. Their framework achieved a DSC of 0. 83
on 307 internal breast cancer patients and maintained
similar performance on two external cohorts, achieving
DSC values of 0. 83
on 307 internal breast cancer patients and maintained
similar performance on two external cohorts, achieving
DSC values of 0. 84 on 206 and 204 patients,
respectively.Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Evaluation Gaps and Future Directions
DSC does not distinguish under-contouring from over-contouring, since a given DSC value can result from either error type or a combination of both. No metric was reported by any of the thirteen studies for sensitivity and precision. p. 7 , 2024;
Maier-Hein et al. The sensitivity and precision
make this distinction: reduced sensitivity indicates
missed tumor tissue and reduced precision indicates
the inclusion of healthy tissue, but neither metric was
reported by any of the thirteen studies. The sensitivity and precision
make this distinction: reduced sensitivity indicates
missed tumor tissue and reduced precision indicates
the inclusion of healthy tissue, but neither metric was
reported by any of the thirteen studies. A geometric score alone may not indicate whether
a clinician would accept the predicted contour. As with geometric accuracy, the aim
should extend to dosimetric equivalence: treatment
plans generated from prompt-driven contours should be
clinically indistinguishable from those generated using
manual contours. The studies included in this review
established the feasibility of prompt-driven segmentation; the logical next step is to determine whether
these geometric improvements correspond to dosimetric
improvements. As with geometric accuracy, the aim
should extend to dosimetric equivalence: treatment
plans generated from prompt-driven contours should be
clinically indistinguishable from those generated using
manual contours. The studies included in this review
established the feasibility of prompt-driven segmentation; the logical next step is to determine whether
these geometric improvements correspond to dosimetric
improvements. No study has reported prospective validation across institutions, imaging protocols,
and clinical workflows for any prompting modality. Generalizability across different RT centers is therefore
the next step towards clinical translation.Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Conclusion of Findings
Models that used a prompt mostly performed better than image-only models, confirming that clinical context adds real value beyond imaging alone. Moving from research to clinical practice will require prospective, multiinstitutional validation; a systematic study of how clinical information should be selected, represented, and used. p. 8 Models that used a
prompt mostly performed better than image-only
models, confirming that clinical context adds real value
beyond imaging alone. However, moving from research
to clinical practice will require prospective, multiinstitutional validation; a systematic study of how clinical information should be selected, represented, and
used, and evaluation that extends beyond geometric
accuracy to dosimetric and clinical outcomes.Where the paper says it
Appendix A Search Strategies
The following search strategies were used to identify eligible studies. Table A.1 shows the search strings used across the three databases. p. 2 This period was selected to capture
the emergence of prompt-driven and multimodal AI
approaches following the introduction of foundation
models and LLMs. Three electronic databases were
searched: PubMed, Embase, and arXiv; the full search
strategies are provided in Appendix A. Review of Prompt-Driven Target Volume Segmentation in Radiotherapy
Table 1
Characteristics of the thirteen included studies. TV: tumor target volume; GTV: gross tumor volume; CTV: clinical target volume;
DSC: Dice Similarity Coefficient; HD95: 95th percentile Hausdorff Distance; MSD: mean surface distance; CMD: center-of-mass
distance; IOU: intersection over union; CTime: contouring time; Multi-site: study evaluated across more than 2 cancer sites.Where the paper says it
Where the paper says it
Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.
Improvements for AI systems
-
Incorporate clinical reasoning into deep learning models by using prompt-driven segmentation to address limitations of image-only models, as
image-only models cannot encode the clinical reasoning that governs TV delineation.
The improved system can now delineate Clinical Target Volume (CTV) whereboundaries are defined by clinical knowledge rather than imaging features alone.
-
Develop multimodal deep learning systems capable of fusing information from different prompt modalities to enhance segmentation accuracy across various cancer sites. This system can combine
geometric prompts, clinical text prompting, and interactive prompting
to achieve better performance than single-modality approaches. -
Implement a system that can systematically compare the performance of different prompting strategies against image-only baselines for specific cancer sites and target volumes. This allows researchers to determine which prompt modality performs best for a given context by comparing
DSC of all thirteen prompt-driven models alongside their image-only baselines.
-
Create an interactive segmentation framework that allows clinicians to refine model predictions during inference, improving accuracy in real-time. This system can utilize
interactive prompting
wherea clinician refines an initial model segmentation through corrective inputs
to achieve higher Dice Similarity Coefficient scores. -
Design a system that extends beyond geometric overlap metrics to evaluate true clinical utility by assessing dosimetric equivalence. This improved AI can determine if
treatment plans generated from prompt-driven contours should be clinically indistinguishable from those generated using manual contours.
Abstract
Accurate target volume (TV) segmentation is essential for radiotherapy (RT) planning but remains time-consuming. Moreover, the final contour depends on the expertise of the radiation oncologist. Although deep learning has improved medical image segmentation, image-only models cannot encode the clinical reasoning that governs TV delineation, particularly for the clinical target volume (CTV), where boundaries are defined by clinical knowledge rather than imaging features alone. Prompt-driven segmentation has recently emerged as a strategy to address this limitation. This work aims to review these approaches for RT TV segmentation with four objectives: (1) compare the different prompting techniques used; (2) compare their performance against image-only baselines; (3) assess how they are evaluated against standard metrics, including the Dice similarity coefficient (DSC), 95th Hausdorff distance, sensitivity, precision, and clinical assessment; and (4) identify the gaps needed for clinical translation. We systematically searched PubMed, Embase, and arXiv for studies incorporating geometric, clinical text, or interactive prompts into deep learning models for RT TV delineation. Thirteen studies met the inclusion criteria, spanning head and neck, breast, prostate, lung, cervix, and rectal cancers. Seven studies used clinical text prompting, four used geometric prompting, and two used interactive prompting; three combined modalities. Reported DSC ranged from 0.67 to 0.93, compared to 0.65 to 0.91 for image-only baselines. None reported sensitivity or precision. Clinical text prompting appears well suited to CTV delineation. Evidence remains limited, and standardized prompt engineering and clinical validation are needed before these approaches can be translated into routine RT practice.
Sources
- TransUNet: Transformers Make Strong Encoders for Medical Image Segmentation
- SAM-Med2D
- EXACT-Net:EHR-guided lung tumor auto-segmentation for non-small cell lung cancer radiotherapy
- nnInteractive: Redefining 3D Promptable Segmentation
- BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation
- Visual Instruction Tuning
- GPT-4 Technical Report
- Learning Transferable Visual Models From Natural Language Supervision
- SAM 2: Segment Anything in Images and Videos
- Generating synthetic computed tomography for radiotherapy: SynthRAD2025 challenge report
- VoxTell: Free-Text Promptable Universal 3D Medical Image Segmentation
Related papers
- HDNet: High-resolution Dual-domain Learning for Spectral Compressive Imaging
- Mask-guided Spectral-wise Transformer for Efficient Hyperspectral Image Reconstruction
- FDIR: Fidelity-Anchored Decompressed Image Restoration for Human and Machine Perception
- A Disease-Centric Vision-Language Foundation Model for Precision Oncology in Kidney Cancer
- CADS: A Comprehensive Anatomical Dataset and Segmentation for Whole-Body Anatomy in Computed Tomography
- Task-Aware Bitrate Thresholds for Codec-Robust Ultrasound Segmentation