From Vision to Context: A Review of Prompt-Driven Target Volume Segmentation in Radiotherapy

arXiv:2611.06570 · eess.IV · Submitted 2026-11-04 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: Electrical Engineering. The day in electrical engineering: signals, systems and the hardware that carries them.

Rosa: We're looking at the paper: "From Vision to Context".

Dev: Here's the gist of the paper. Accurate target volume (TV) segmentation is essential for radiotherapy (RT) planning but remains time-consuming and dependent on expert knowledge This work aims to review these approaches for RT TV segmentation with four objectives.

Rosa: Manual TV delineation remains one of the most time-consuming and error-prone steps in the RT workflow This error-proneness is most evident in interobserver variability (IOV) A large body of evidence has shown that IOV in TV contouring is a major source of geometric uncertainty in RT planning.

Dev: Substantial IOV has been reported across multiple cancer sites, including lung, prostate, and esophageal cancer.

Rosa: IOV arises from multiple factors, including radiation oncologist expertise, image quality, and the inherent subjectivity in interpreting ambiguous tumor boundaries on medical images.

Paper summary: Rosa: So we're looking at this paper, "From Vision to Context: A Review of Prompt-Driven Target Volume Segmentation in Radiotherapy." Basically, the main point is that accurate target volume segmentation is super important for radiotherapy planning but it still takes too much time and depends on the radiation oncologist's expertise.

Dev: Yeah, and image-only models just can't capture that clinical reasoning needed for the clinical target volume or CTV, where the boundaries are defined by things a doctor knows rather than just what’s visible in a picture.

Rosa: Exactly. And this paper reviews different ways people are trying to fix that limitation using prompt-driven segmentation as a strategy to bring in that missing context.

Taro: So what's the big question here? What exactly is this review comparing across all these prompt methods?

Dev: The authors set four objectives for this review. They want to compare the different prompting techniques being used, see how they actually perform against those image-only baselines, look at how they are evaluated using standard metrics like Dice similarity coefficient and 95th Hausdorff distance, sensitivity and precision, and finally figure out what gaps we need to fill for actual use in a clinic <ref:2611.06570#pg1>.

Rosa: It’s interesting because the search strategy they used was pretty thorough. They looked across PubMed, Embase, and arXiv for studies that used geometric prompts, clinical text inputs, or interactive prompting into deep learning models for radiotherapy target volume delineation.

Dev: That search pulled in thirteen studies overall that covered a range of cancers like head and neck, breast, prostate cancer, lung cancer and rectal cancer. Out of those thirteen studies they found seven that used clinical text prompting four that used geometric prompting and two that used interactive prompting, plus three studies combined both modalities.

Rosa: The performance numbers they reported are pretty encouraging but there are some caveats. They mentioned reported Dice similarity coefficients ranged from zero point sixty-seven to zero point ninety-three which is compared against the image-only baselines scoring from zero point sixty-five to zero point ninety-one.

Dev: But then they also noted that none of the studies reported sensitivity or precision metrics for these segmentation models. They focused mostly on that Dice score and the Hausdorff distance measures like HD95.

Taro: So if we only look at those numbers, it seems like prompt-driven models are generally better than image-only ones, but what does that actually mean for a patient getting treatment?

Rosa: The paper points out that clinical text prompting seems to be particularly well suited for delineating the CTV. They also noted that geometric prompting achieved a mean Dice score of zero point eighty-four on one set of test patients from four institutions, which was better than five image-only models.

Dev: Clinical text prompting hit a Dice score of zero point eighty-three on three hundred seven internal breast cancer patients and kept similar performance on two other external groups at scores of zero point eighty-two and zero point eighty-four respectively. That suggests a lot of value from the clinical context being fed into the model.

Rosa: The paper is careful about what it says, though. It highlights that the Dice score itself doesn't tell you if the contouring is too big or too small; a certain Dice value could mean under-contouring or over-contouring, or both happening at once.

Dev: And they explicitly state that no metric was reported for sensitivity and precision, which is a major gap when we think about clinical safety and how much error we can tolerate.

Taro: So what’s the next logical step according to the authors? What do they suggest is needed before this technology can actually move into routine radiotherapy practice?

Rosa: They say that geometric scores alone might not tell you if a clinician would actually accept the predicted contour. So, it seems like the next step is figuring out if these geometric improvements translate into actual dosimetric improvements for the patient.

Dev: And they point to generalizability across different radiotherapy centers as the next big thing for clinical translation because that’s where things often get messy in real practice.

Taro: So, we see models doing better than just looking at pictures, but we still have to figure out how to make sure those geometric improvements actually result in a better physical dose delivered to the tumor.

Rosa: That’s the core of it. The paper concludes that moving this from research to practice will require prospective, multiinstitutional validation and a systematic study on how clinical information should be selected, represented, and used in these models.

Dev: So for anyone listening who cares about how this changes the workflow, it means we need more than just better segmentation scores; we need real-world testing showing it works reliably across different hospitals.

Conclusion: Rosa: So we're wrapping up this review of "From Vision to Context: A Review of Prompt-Driven Target Volume Segmentation in Radiotherapy."

Dev: That paper looks at how we can get better target volume segmentation when we use deep learning, focusing on adding clinical context through prompting.

Rosa: Right, it looks at the different ways people are asking the model for extra information—whether that's geometry, text from patient reports, or interactive refinement.

Dev: The authors found that these prompt-driven models generally do better than just using image data alone for tasks like delineating the clinical target volume.

Rosa: They looked at thirteen studies across many cancer types and saw that clinical text prompting seems particularly useful for defining the actual CTV boundaries, which is where doctor knowledge matters most.

Dev: The results show Dice similarity coefficients in that range of zero point sixty-seven to zero point ninety-three when compared against image-only models.

Rosa: But here's the thing: they also pointed out that this Dice score doesn't tell you if the contour is over or under delineated; it could be both at once.

Dev: And they didn't report sensitivity or precision metrics, which means we don't know how often the model actually gets it right or what kind of error we’re dealing with.

Rosa: So the big question left hanging in that review is whether these geometric improvements translate into actual physical dose improvements for the patient.

Dev: That’s where things get tricky; moving this from a research paper to routine practice requires prospective testing across different hospitals.

Bakpen Kombata, Guus Grimbergena, Maarten L. Terpstrab, Gert J. Meijera, Lois A. Daamena

Department of Radiation Oncology, University Medical Center Utrecht · Computational Imaging Group for MR Diagnostics and Therapy, Center for Image Sciences, University Medical Center Utrecht

eess.IV

Submitted: 2026-11-04

Updated: 2026-11-04

Comments: Submitted to Computerized Medical Imaging and Graphics

License: http://creativecommons.org/licenses/by/4.0/

Importance score: 81/100

The gist: Accurate target volume (TV) segmentation is essential for radiotherapy (RT) planning but remains time-consuming and dependent on expert knowledge This work aims to review these approaches for RT TV…

Key concepts

Target Volume (TV) Segmentation
This is the process of accurately outlining the area within a patient's body that needs to receive radiation. Accurate segmentation is crucial for planning radiotherapy, but manual contouring is time-consuming and varies between different doctors, leading to geometric uncertainty in treatment plans.
Prompt-Driven Models (PDMs)
These are deep learning models enhanced by providing external information—a 'prompt'—beyond the medical image itself. This prompt can be geometric data, clinical text like patient reports, or interactive clinician input. PDMs aim to incorporate this crucial clinical reasoning into the segmentation process.
Clinical Text Prompting
This technique involves feeding structured or unstructured clinical information, such as TNM staging or contouring guidelines, directly into the segmentation model alongside the image data. The review found this modality particularly effective for defining Clinical Target Volumes (CTV).
Dice Similarity Coefficient (DSC)
DSC is a standard metric used to measure how well a predicted segmentation mask matches the ground truth (the expert contour). A higher DSC value indicates better accuracy in the segmentation, ranging from 0.67 to 0.93 in this review.

Terminology

Summary

Accurate target volume (TV) segmentation is essential for radiotherapy (RT) planning but remains time-consuming and dependent on expert knowledge This work aims to review these approaches for RT TV segmentation with four objectives. p. 0

Where the paper says it

Meijera and Lois A. Daamena aDepartment of Radiation Oncology, University Medical Center Utrecht, Utrecht, The Netherlands bComputational Imaging Group for MR Diagnostics and Therapy, Center for Image Sciences, University Medical Center Utrecht, Utrecht, The Netherlands A R T I C L E I N F O Keywords: Radiotherapy target volume deep learning prompt-driven segmentation clinical text prompting A B S T R A C T Accurate target volume (TV) segmentation is essential for radiotherapy (RT) planning but remains time-consuming.

Introduction to the Problem

Manual TV delineation remains one of the most time-consuming and error-prone steps in the RT workflow This error-proneness is most evident in interobserver variability (IOV) A large body of evidence has shown that IOV in TV contouring is a major source of geometric uncertainty in RT planning. Substantial IOV has been reported across multiple cancer sites, including lung, prostate, and esophageal cancer. p. 0

Where the paper says it

, 2023; Segedin and Petric, 2016). Substantial IOV has been reported across multiple cancer sites, including lung, prostate, and esophageal cancer (Guzene et al.

IOV arises from multiple factors, including radiation oncologist expertise, image quality, and the inherent subjectivity in interpreting ambiguous tumor boundaries on medical images. p. 0
Where the paper says it

, 2025; Peng et al. IOV arises from multiple factors, including radiation oncologist expertise, image quality, and the inherent subjectivity in interpreting ambiguous tumor boundaries on medical images (Guzene et al.

The Need for Prompt-Driven Models

Image-only models cannot encode the clinical reasoning that governs TV delineation, particularly for the clinical target volume (CTV), where boundaries are defined by clinical knowledge rather than imaging features alone. This limitation is not expected to be resolved by developing larger architectures or training on more image data alone. p. 1

Where the paper says it

As none of these features are encoded in the imaging, a segmentation model trained only on image data can learn only the relationship between image appearance and the contours provided during training, without access to the clinical information that determines the CTV extent. This limitation is not expected to be resolved by developing larger architectures or training on more image data alone.

This challenge has motivated the development of prompt-driven models (PDMs): segmentation models that incorporate information that cannot be derived from the image alongside the imaging data. p. 1
Where the paper says it

This limitation is not expected to be resolved by developing larger architectures or training on more image data alone. This challenge has motivated the development of prompt-driven models (PDMs): segmentation models that incorporate information that cannot be derived from the image alongside the imaging data.

Depending on the clinical context, this information, referred to as a prompt, may be geometric, textual, or interactive. p. 1
Where the paper says it

This challenge has motivated the development of prompt-driven models (PDMs): segmentation models that incorporate information that cannot be derived from the image alongside the imaging data. Depending on the clinical context, this information, referred to as a prompt, may be geometric, textual, or interactive.

Classification of Prompt Modalities

Studies were classified according to the type of prompt incorporated into the segmentation process. Three prompt modalities were identified (Figure 1):

(1) geometric prompting, in which spatial inputs such as points, bounding boxes, or binary masks indicate the approximate location of the TV.

(2) clinical text prompting, in which structured clinical data such as TV location and TNM staging, or unstructured information such as clinical reports and contouring guidelines, are encoded as text and fused with imaging features. p. 1

Where the paper says it

Prompt Modality Classification Studies were classified according to the type of prompt incorporated into the segmentation process. Three prompt modalities were identified (Figure 1): (1) geometric prompting, in which spatial inputs such as points, bounding boxes, or binary masks indicate the approximate location of the TV; (2) clinical text prompting, in which structured clinical data such as TV location and TNM staging, or unstructured information such as clinical reports and contouring guidelines, are encoded as text and fused with imaging features; and (3) interactive prompting, in which a clinician refines an initial model segmentation through corrective Kombat et al.

(3) interactive prompting, in which a clinician refines an initial model segmentation through corrective inputs. p. 1

Where the paper says it

Prompt Modality Classification Studies were classified according to the type of prompt incorporated into the segmentation process. Three prompt modalities were identified (Figure 1): (1) geometric prompting, in which spatial inputs such as points, bounding boxes, or binary masks indicate the approximate location of the TV; (2) clinical text prompting, in which structured clinical data such as TV location and TNM staging, or unstructured information such as clinical reports and contouring guidelines, are encoded as text and fused with imaging features; and (3) interactive prompting, in which a clinician refines an initial model segmentation through corrective Kombat et al.

Performance Comparison Across Modalities

Reported DSC ranged from 0.67 to 0.93, compared to 0. p. 0

Where the paper says it

Reported DSC ranged from 0. 93, compared to 0.

65 to 0.91 for image-only baselines. p. 0
Where the paper says it

Prompt-driven segmentation has recently emerged as a strategy to address this limitation. This work aims to review these approaches for RT TV segmentation with four objectives: (1) compare the different prompting techniques used; (2) compare their performance against image-only baselines; (3) assess how they are evaluated against standard metrics, including the Dice similarity coefficient (DSC), 95th Hausdorff distance, sensitivity, precision, and clinical assessment; and (4) identify the gaps needed for clinical translation.

Clinical text prompting appears well suited to CTV delineation. p. 0
Where the paper says it

None reported sensitivity or precision. Clinical text prompting appears well suited to CTV delineation.

Geometric prompting achieved a mean DSC of 0. p. 3
Where the paper says it

Six geometric prompting strategies, ranging from a single center point to three orthogonal binary masks, were simulated from the reference GTV contours. Across 537 patients from six institutions, nnInteractive achieved the highest median DSC, increasing from 0.

84 on 120 test patients from four institutions, outperforming five image-only models. p. 3
Where the paper says it

Their framework achieved a mean DSC of 0. 84 on 120 test patients from four institutions, outperforming five image-only models.

Clinical text prompting achieved a DSC of 0. p. 7
Where the paper says it

One general-purpose segmentation model not built specifically for RT has used this approach, and despite being trained on a very large and varied dataset, achieved lower accuracy than the domain-specific models in this review (Rokuss et al. This suggests that the flexibility of free-text prompting may come at the cost of accuracy without training tailored to the specific clinical task.

83 on 307 internal breast cancer patients and maintained similar performance on two external cohorts, achieving DSC values of 0. p. 4
Where the paper says it

Their framework achieved a DSC of 0. 83 on 307 internal breast cancer patients and maintained similar performance on two external cohorts, achieving DSC values of 0.

82 and 0.84 on 206 and 204 patients, respectively. p. 4
Where the paper says it

83 on 307 internal breast cancer patients and maintained similar performance on two external cohorts, achieving DSC values of 0. 84 on 206 and 204 patients, respectively.

Evaluation Gaps and Future Directions

DSC does not distinguish under-contouring from over-contouring, since a given DSC value can result from either error type or a combination of both. No metric was reported by any of the thirteen studies for sensitivity and precision. p. 7

Where the paper says it

, 2024; Maier-Hein et al. The sensitivity and precision make this distinction: reduced sensitivity indicates missed tumor tissue and reduced precision indicates the inclusion of healthy tissue, but neither metric was reported by any of the thirteen studies.

A geometric score alone may not indicate whether a clinician would accept the predicted contour. p. 7
Where the paper says it

The sensitivity and precision make this distinction: reduced sensitivity indicates missed tumor tissue and reduced precision indicates the inclusion of healthy tissue, but neither metric was reported by any of the thirteen studies. A geometric score alone may not indicate whether a clinician would accept the predicted contour.

The logical next step is to determine whether these geometric improvements correspond to dosimetric improvements. p. 7
Where the paper says it

As with geometric accuracy, the aim should extend to dosimetric equivalence: treatment plans generated from prompt-driven contours should be clinically indistinguishable from those generated using manual contours. The studies included in this review established the feasibility of prompt-driven segmentation; the logical next step is to determine whether these geometric improvements correspond to dosimetric improvements.

p. 7
Where the paper says it

As with geometric accuracy, the aim should extend to dosimetric equivalence: treatment plans generated from prompt-driven contours should be clinically indistinguishable from those generated using manual contours. The studies included in this review established the feasibility of prompt-driven segmentation; the logical next step is to determine whether these geometric improvements correspond to dosimetric improvements.

Generalizability across different RT centers is therefore the next step towards clinical translation. p. 8
Where the paper says it

No study has reported prospective validation across institutions, imaging protocols, and clinical workflows for any prompting modality. Generalizability across different RT centers is therefore the next step towards clinical translation.

Conclusion of Findings

Models that used a prompt mostly performed better than image-only models, confirming that clinical context adds real value beyond imaging alone. Moving from research to clinical practice will require prospective, multiinstitutional validation; a systematic study of how clinical information should be selected, represented, and used. p. 8

Where the paper says it

Models that used a prompt mostly performed better than image-only models, confirming that clinical context adds real value beyond imaging alone. However, moving from research to clinical practice will require prospective, multiinstitutional validation; a systematic study of how clinical information should be selected, represented, and used, and evaluation that extends beyond geometric accuracy to dosimetric and clinical outcomes.

The logical next step is to determine whether these geometric improvements correspond to dosimetric improvements.

Appendix A Search Strategies

The following search strategies were used to identify eligible studies. Table A.1 shows the search strings used across the three databases. p. 2

Where the paper says it

This period was selected to capture the emergence of prompt-driven and multimodal AI approaches following the introduction of foundation models and LLMs. Three electronic databases were searched: PubMed, Embase, and arXiv; the full search strategies are provided in Appendix A.

The PubMed search string included (“radiotherapy”[ti/ab] OR “radiation therapy”[ti/ab] OR “radiation oncology”[ti/ab]) AND (“target volume”[ti/ab] OR “clinical target volume”[ti/ab] OR “CTV”[ti/ab] OR “gross tumor volume”[ti/ab]) AND (“segmentation”[ti/ab] OR “delineation”[ti/ab] OR “contouring”[ti/ab]) AND (“foundation model”[ti/ab] OR “multimodal”[ti/ab] OR “vision-language”[ti/ab] OR “large language model”[ti/ab] OR “LLM”[ti/ab] OR “prompt”[ti/ab] OR “SAM”[ti/ab] OR “segment anything”[ti/ab]) AND “2022/01/01”[Date]: “2026/04/08”[Date]. p. 5
Where the paper says it

Review of Prompt-Driven Target Volume Segmentation in Radiotherapy Table 1 Characteristics of the thirteen included studies. TV: tumor target volume; GTV: gross tumor volume; CTV: clinical target volume; DSC: Dice Similarity Coefficient; HD95: 95th percentile Hausdorff Distance; MSD: mean surface distance; CMD: center-of-mass distance; IOU: intersection over union; CTime: contouring time; Multi-site: study evaluated across more than 2 cancer sites.

Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.

Improvements for AI systems

  1. Incorporate clinical reasoning into deep learning models by using prompt-driven segmentation to address limitations of image-only models, as image-only models cannot encode the clinical reasoning that governs TV delineation. The improved system can now delineate Clinical Target Volume (CTV) where boundaries are defined by clinical knowledge rather than imaging features alone.

  2. Develop multimodal deep learning systems capable of fusing information from different prompt modalities to enhance segmentation accuracy across various cancer sites. This system can combine geometric prompts, clinical text prompting, and interactive prompting to achieve better performance than single-modality approaches.

  3. Implement a system that can systematically compare the performance of different prompting strategies against image-only baselines for specific cancer sites and target volumes. This allows researchers to determine which prompt modality performs best for a given context by comparing DSC of all thirteen prompt-driven models alongside their image-only baselines.

  4. Create an interactive segmentation framework that allows clinicians to refine model predictions during inference, improving accuracy in real-time. This system can utilize interactive prompting where a clinician refines an initial model segmentation through corrective inputs to achieve higher Dice Similarity Coefficient scores.

  5. Design a system that extends beyond geometric overlap metrics to evaluate true clinical utility by assessing dosimetric equivalence. This improved AI can determine if treatment plans generated from prompt-driven contours should be clinically indistinguishable from those generated using manual contours.

Abstract

Accurate target volume (TV) segmentation is essential for radiotherapy (RT) planning but remains time-consuming. Moreover, the final contour depends on the expertise of the radiation oncologist. Although deep learning has improved medical image segmentation, image-only models cannot encode the clinical reasoning that governs TV delineation, particularly for the clinical target volume (CTV), where boundaries are defined by clinical knowledge rather than imaging features alone. Prompt-driven segmentation has recently emerged as a strategy to address this limitation. This work aims to review these approaches for RT TV segmentation with four objectives: (1) compare the different prompting techniques used; (2) compare their performance against image-only baselines; (3) assess how they are evaluated against standard metrics, including the Dice similarity coefficient (DSC), 95th Hausdorff distance, sensitivity, precision, and clinical assessment; and (4) identify the gaps needed for clinical translation. We systematically searched PubMed, Embase, and arXiv for studies incorporating geometric, clinical text, or interactive prompts into deep learning models for RT TV delineation. Thirteen studies met the inclusion criteria, spanning head and neck, breast, prostate, lung, cervix, and rectal cancers. Seven studies used clinical text prompting, four used geometric prompting, and two used interactive prompting; three combined modalities. Reported DSC ranged from 0.67 to 0.93, compared to 0.65 to 0.91 for image-only baselines. None reported sensitivity or precision. Clinical text prompting appears well suited to CTV delineation. Evidence remains limited, and standardized prompt engineering and clinical validation are needed before these approaches can be translated into routine RT practice.

Sources

Related papers