CADS: A Comprehensive Anatomical Dataset and Segmentation for Whole-Body Anatomy in Computed Tomography
Listen
Radio episode about this paper
Transcript
Introduction to the show: ident: Electrical Engineering. The day in electrical engineering: signals, systems and the hardware that carries them.
Rosa: We're looking at the paper: "CADS: A Comprehensive Anatomical Dataset and Segmentation for Whole-Body Anatomy in Computed Tomography".
Dev: Here's the gist of the paper. Accurate delineation of anatomical structures in volumetric Computed Tomography (CT) scans is crucial for diagnosis and treatment planning, yet current approaches often suffer from a fragmented landscape of incompatible models with varying performance protocols.
Rosa: Foundational segmentation models designed to process multiple organs offer a holistic view, but robust clinical deployment requires comprehensive training data that addresses data heterogeneity and anatomical coverage—areas where existing whole-body approaches are deficient.
Dev: This work introduces CADS (Comprehensive Anatomical Dataset and Segmentation for Whole-Body Anatomy in Computed Tomography), an open-source framework prioritizing the systematic integration, standardization, and labeling of heterogeneous, cross-institutional, and cross-vendor CT data sources for whole-body CT segmentation.
Rosa: The core contribution is a large-scale dataset comprising 22,022 CT volumes with complete annotations for 167 anatomical structures.
Title and authors: Rosa: So we're looking at this paper today called "CADS: A Comprehensive Anatomical Dataset and Segmentation for Whole-Body Anatomy in Computed Tomography". It sounds like a big deal for getting AI to actually understand the whole body from CT scans.
Dev: Yeah, it's about building a dataset and a segmentation model that can handle all those structures across the entire body. The authors are Xu, Amiranashvili, Navarro, Fritsak, Hamamci, Shitmann, Wittmann, Christman-Christ3, de la Rosa1, Deseoe1, Graf8.
Rosa: What this means for us is that we're finally looking at a way to get comprehensive anatomical delineation instead of just looking at one organ in isolation. It moves us toward a holistic view.
Dev: Exactly. The authors are aiming to advance robust AI solutions in radiology and make comprehensive anatomical analysis accessible to clinicians and researchers alike, as mentioned on page two > <ref:2507.22953#pg1>
The paper's summary: Rosa: So what's the actual substance of this CADS paper? Basically, they're presenting an open-source framework. It’s not just a single model; it’s a whole system for organizing and labeling data from all over.
Dev: They are focusing on integrating heterogeneous, cross-institutional, and cross-vendor CT data sources systematically. The core contribution is this large dataset they put together.
Rosa: That dataset has twenty-two thousand twenty-two CT volumes with complete annotations for one hundred sixty-seven different anatomical structures <ref:2507.22953#pg3,22,022 CT volumes with>. That’s a huge scale compared to what’s out there right now.
Dev: It is eighteen times more scans than existing collections and sixty percent more distinct anatomical targets, which is a significant increase in coverage. They are using pseudo-labeling grounded in self-training through iterative model refinement, shape-guided qualification of those labels >
The paper's improvements: Rosa: The authors talk about how they built this system. It’s a multi-stage pipeline to go from raw data to the final segmentation model.
Dev: They started with training initial region-specific models on nine anatomical regions, which cover all those one hundred sixty-seven target structures, and then generating pseudo-labels across all twenty-two thousand twenty-two volumes > <ref:2507.22953#pg3>
Rosa: Then they have this quality control step where they use neural implicit functions and shape priors to filter out the unreliable pseudo-labels. That’s a smart way to clean up the noise.
Dev: And then they use a multi-strategy label optimization process. They combine three complementary models, or flavors, each trained with different data characteristics, to create the final CADS dataset >
Rosa: Finally, they train the main CADS model using customized class balancing strategies to make sure every structure gets attention. It’s a very detailed approach to building this comprehensive system.
Conclusion: Dev: So putting it all together, the paper presents the CADS framework, which includes both the dataset and a robust segmentation model suite trained on it. The whole point is accessibility and scale for whole-body analysis.
Rosa: It really shows how much more data and better organization we need to move beyond looking at just one organ at a time in CT scans. The performance metrics are pretty impressive across the board, with an overall Dice score of eighty-five point eight seven percent across all one hundred sixty-seven structures >
Dev: Especially for things like the heart, they got average Dice improvements of nine point two percent for myocardium and seven point five percent for pulmonary artery structures >
Rosa: It’s a lot of data diversity—spanning forty sources in sixteen countries—which means the model should generalize pretty well to different imaging protocols and patient groups.
Dev: But we have to remember what they admit about their process. They mention that closed-source models and deliverables hinder reproducibility, which is a real challenge for adoption >
Rosa: And they also pointed out accessibility barriers, like the absence of user-friendly tools, which can stop healthcare practitioners from actually integrating these solutions into clinical workflows >
Dev: So while the results on performance are strong across various regions—like the Liver getting a mean Dice score of ninety-six point nine four plus two point zero zero—the real hurdle is making sure everyone can use this tool reliably in a real hospital setting >
Department of Quantitative Biomedicine University of Zurich · ETH AI Center ETH Zurich · Department of Radiation Oncology University Hospital and University of Zurich · Faculty of Medicine University of Zurich · School of Computation Information and Technology Technical University of Munich · Department of Diagnostic and Interventional Neuroradiology School of Medicine and Health Technical University of Munich Departmental Imaging Institute in Artificial Intelligence in Medicine TUM School of Medicine and Health Institute for Radiation Medicine Helmholtz Zentrum München Departmental Imaging Institute in Artificial Intelligence in Medicine Klinikum rechts der Isar TUM School of Medicine and Health German Consortium for Translational Cancer Research Partner Site Munich University Hospital Essen Institute for Interventional and Diagnostic Radiology Pediatric Radiology and Neuroradiology University Medical Center Rostock Medical University of Vienna Department of Biomedical Imaging and Image-guided Therapy Medical University of Vienna Department of Radiology University Hospital Basel Departmental Imaging Institute in Artificial Intelligence in Medicine Essen Institute for Diagnostic and Interventional Radiology Pediatric Radiology and Neuroradiology University Medical Center Rostock Departmental Imaging Institute in Artificial Intelligence in Medicine Essen
eess.IV, cs.CV
Submitted: 2025-07-29
Updated: 2026-11-10
Journal ref: Nature Communications (2026)
DOI: 10.1038/s41467-026-78732-1
Code: https://github.com/murong-xu/CADS
License: http://creativecommons.org/licenses/by/4.0/
Importance score: 100/100
The gist: Accurate delineation of anatomical structures in volumetric Computed Tomography (CT) scans is crucial for diagnosis and treatment planning…
Key concepts
- CADS Dataset
- This is a massive collection of 22,022 CT volumes with complete labels for 167 different anatomical structures. It combines data from many different hospitals and sources to create a comprehensive resource for training AI models.
- CADS-model
- This is the fully automated segmentation model suite trained on the CADS dataset. It uses a 3D U-Net architecture and is designed to segment all 167 target structures consistently across various anatomical regions, like the heart or bones.
- Multi-strategy Label Optimization
- This process involves training several complementary models ('flavors') on different data subsets. The best model for each structure is then selected based on performance metrics, ensuring that the final dataset has high-quality labels for all structures.
- Performance-driven Flavor Selection
- When creating the final CADS dataset, each anatomical structure is evaluated using Dice Similarity Coefficient and Hausdorff distance. The label from the flavor that performs best for that specific structure is chosen as the definitive label.
Terminology
Summary
Accurate delineation of anatomical structures in volumetric Computed Tomography (CT) scans is crucial for diagnosis and treatment planning, yet current approaches often suffer from a fragmented landscape of incompatible models with varying performance protocols. p. 0 Accurate delineation of anatomical structures in volumetric Computed Tomography
(CT) scans is crucial for diagnosis and treatment planning. While AI has advanced automated
segmentation, current approaches typically target individual structures, creating a fragmented
landscape of incompatible models with varying performance and disparate evaluation protocols. Foundational segmentation models designed to process more organs address these limitations by
providing a holistic anatomical view through a single model. Yet, robust clinical deployment demands comprehensive training data, which is lacking in existing whole-body approaches – both
in terms of data heterogeneity and, more importantly, anatomical coverage.Where the paper says it
Where the paper says it
This work introduces CADS (Comprehensive Anatomical Dataset and Segmentation for Whole-Body Anatomy in Computed Tomography), an open-source framework prioritizing the systematic integration, standardization, and labeling of heterogeneous, cross-institutional, and cross-vendor CT data sources for whole-body CT segmentation. p. 0 Yet, robust clinical deployment demands comprehensive training data, which is lacking in existing whole-body approaches – both
in terms of data heterogeneity and, more importantly, anatomical coverage. In this work, rather
than pursuing incremental optimizations in model architecture, we present CADS, an open-source
framework that prioritizes the systematic integration, standardization, and labeling of heterogeneous cross-institutional and cross-vendor data sources for whole-body CT segmentation. In this work, rather
than pursuing incremental optimizations in model architecture, we present CADS, an open-source
framework that prioritizes the systematic integration, standardization, and labeling of heterogeneous cross-institutional and cross-vendor data sources for whole-body CT segmentation. At its
core is a large-scale dataset of 22,022 CT volumes with complete annotations for 167 anatomical
structures, representing a significant advancement in both scale and coverage, with 18 times more
scans than existing collections and 60% more distinct anatomical targets. In this work, rather
than pursuing incremental optimizations in model architecture, we present CADS, an open-source
framework that prioritizes the systematic integration, standardization, and labeling of heterogeneous cross-institutional and cross-vendor data sources for whole-body CT segmentation. At its
core is a large-scale dataset of 22,022 CT volumes with complete annotations for 167 anatomical
structures, representing a significant advancement in both scale and coverage, with 18 times more
scans than existing collections and 60% more distinct anatomical targets.Where the paper says it
Where the paper says it
Where the paper says it
The CADS framework consists of two main components: the CADS-dataset itself and the CADS-model, a robust, fully automated segmentation model suite trained on this dataset. p. 2 The collection spans more than 100 imaging facilities across
16 countries, capturing a wide spectrum of clinical variability. CADS-model: A robust, fully automated segmentation model suite trained on the CADS-dataset and
thoroughly validated, capable of segmenting 167 structures from head to knee across diverse anatomical
systems.Where the paper says it
flavors
) trained with varying data characteristics to create the assembled CADS-dataset; and (4) final CADS-model training with customized class balancing strategies.
The CADS-model utilizes a 3D full-resolution U-Net architecture with residual connections in the encoder path, selected for its robust performance and efficient runtime compatibility with nnU-Net's L configuration. The dataset is highly diverse, spanning 40 sources across 16 countries (including public challenges, TCIA collections, and newly acquired clinical data), encompassing normal and pathological cases across various imaging protocols. p. 4 Our data collection strategy systematically consolidates CT scans and annotations from three main
sources: (1) large-scale public archives including TotalSegmentator [27] (1,203 annotated volumes with
comprehensive coverage of 104 anatomical structures), which provides a foundational basis for whole-body
segmentation, along with the National Lung Screening Trial (NLST [40], 7,172 scans), CT-RATE [41]
(3,134 scans), and AbdomenCT-1K [42] (1,062 scans); (2) specialized annotation collections from medical
imaging challenges (via Grand Challenge [36] and MICCAI [37]) and focused research studies (such as
Han-Seg [43] for head-and-neck OARs, SAROS [44] for broad body context structures, and various oncology studies from TCIA [35]); and (3) newly contributed images and annotations, comprising manual
annotations of critical OARs on the VISCERAL [12] dataset, 484 newly acquired anonymized head CTs,
and 586 triphasic contrast-enhanced abdominal CT scans, which together strengthen coverage of cranial anatomy and multi-phase contrast imaging. This systematic integration unifies previously dispersed
public datasets from medical imaging challenges and research initiatives, creating a unified resource that
standardizes annotations while contributing new clinical data to the community.Where the paper says it
The framework demonstrates superior performance over state-of-the-art approaches when evaluated across 18 public datasets and an independent real-world hospital cohort. p. 0 Through comprehensive evaluation across 18 public datasets and an independent real-world hospital cohort, we demonstrate advantages over state-of-the-art approaches. Notably, thorough testing of the model’s performance in segmentation tasks from radiation onarXiv:2507. CADS 7
Analysis across major anatomical systems further underscores the importance of training on the larger
and more comprehensive CADS-dataset. For cardiovascular structures, we observe significant average Dice
improvements in myocardium (+9. 4 Xu et al. Right ribs 1-12
Left ribs 1-12
Kidney R/L
Upper lobe of lung R/L
Middle lobe of lung R
Lower lobe of lung R/L
Adrenal gland R/L
Inferior vena cava
Portal vein and splenic vein
Liver
Aorta
Spleen
Gallbladder
Stomach
Pancreas
Brain
Face
Colon
Trachea
Urinary bladder
Duodenum
Small intestine
Esophagus
Myocardium
Heart atrium R/L
Heart ventricle R/L
Common iliac artery R/L
Common iliac vein R/L
Pulmonary artery
Seminal vesicle
Sigmoid colon
Colostomy bag
Mammary gland R/L
Psoas major muscle R/L
Rectus abdominis muscle R/L
Heart
Larynx
Rectum
Prostate
Sternum
Spinal canal
Bones
Muscle
Spinal cord
Pericardium
Mediastinum
Pituitary gland
Cervical vertebrae C1-C7
Thoracic vertebrae T1-T12
Lumbar vertebrae L1-L5
Cochlear R/L
Optic nerve R/L
Carotid artery R/L
Parotid gland R/L
Arytenoid cartilage
Cervical esophagus
Lacrimal gland R/L
Submandibular gland R/L
Anterior eyeball segment R/L
Posterior eyeball segment R/L
Glottis
Thyroid
Mandible
Pituitary gland
Buccal mucosa
Cricopharyngeus
Both lips
Oral cavity
Brainstem
Supraglottis
Optic chiasm
Thoracic cavity
Abdominal cavity
Gland structure
Subcutaneous tissue
Prosthetic breast implant
White matter
Spongy bone
Compact bone
Muscle of head
Cerebrospinal fluid
Scalp
Blood
Eyeball
Gray matter
Iliopsoas muscle R/L
Gluteus minius muscle R/L
Gluteus medius muscle R/L
Gluteus maximus muscle R/L
Deep muscle of back R/L
Sacrum
Hip R/L
Femur R/L
Clavicle R/L
Scapula R/L
Humerus R/L
Fig. 2%), pulmonary artery (+7. 5%), and heart chambers (+5. 5%), and heart chambers (+5. 3%)
Skeletal structures demonstrate high precision in both sequential and individual elements: ribs achieve
89. 3%)
Skeletal structures demonstrate high precision in both sequential and individual elements: ribs achieve
89. 2% with improvements in 22/24 structures (mean +2. Yet current resources
face several fundamental constraints that limit the performance ceiling of models trained on them. These challenges include: Limited anatomical coverage – many approaches focus on specific regions
or restricted structures, falling short of comprehensive whole-body analysis [31,32]; Insufficient data diversity – models developed on small-scale or single-source datasets show limited generalizability across
institutions, populations, and imaging protocols [27,32,33,30,34]; Inconsistent annotation quality – existing automated labeling approaches introduce known systematic errors in complex anatomical structures
like ribs and vertebrae [27], which models trained on such data learn and propagate; Human-dependent
workflows – current approaches often rely on human-in-the-loop revisions during model training and
dataset construction, limiting scalability and increasing operational costs [33,31]; Inadequate validation
– evaluations are mostly conducted on internal or same-distribution datasets, with limited external testing, leaving real-world robustness unexplored [32,33,30,34]; Limited availability – some solutions remain 3%)
Skeletal structures demonstrate high precision in both sequential and individual elements: ribs achieve
89. 2% with improvements in 22/24 structures (mean +2. The CADS-model extends beyond these mutual targets to segment 48 additional structures (shown
by green cross markers in Figure 3), including critical sensory organs, body cavities, and reproductive
structures. Across all 167 target structures, the CADS-model achieves an overall Dice score of 85. Complete performance metrics beyond
Dice scores are provided in Online Method Section 5. Through systematic evaluation, the CADS-model trained on the CADS-dataset demonstrates consistent performance across diverse anatomical structures, with robust generalization across clinical settings. Results demonstrate the advantages of training on the larger, more diverse CADS-dataset over models
limited to the TotalSegmentator dataset. For the 119 mutual targets, the CADS-model achieves a mean
Dice score of 90. Unlike
natural image datasets from RGB cameras that can contain millions of samples [23,24,25,26], medical
datasets face two key limitations: they are orders of magnitude smaller and rarely annotated. Our analysis of 40 diverse sources reveals that only 21% of publicly available CT scans contain any annotations,
with annotations typically confined to a single organ system (Table 1). Results demonstrate the advantages of training on the larger, more diverse CADS-dataset over models
limited to the TotalSegmentator dataset. For the 119 mutual targets, the CADS-model achieves a mean
Dice score of 90. Unlike
natural image datasets from RGB cameras that can contain millions of samples [23,24,25,26], medical
datasets face two key limitations: they are orders of magnitude smaller and rarely annotated. Our analysis of 40 diverse sources reveals that only 21% of publicly available CT scans contain any annotations,
with annotations typically confined to a single organ system (Table 1). Results demonstrate the advantages of training on the larger, more diverse CADS-dataset over models
limited to the TotalSegmentator dataset. For the 119 mutual targets, the CADS-model achieves a mean
Dice score of 90. Unlike
natural image datasets from RGB cameras that can contain millions of samples [23,24,25,26], medical
datasets face two key limitations: they are orders of magnitude smaller and rarely annotated. Our analysis of 40 diverse sources reveals that only 21% of publicly available CT scans contain any annotations,
with annotations typically confined to a single organ system (Table 1). Results demonstrate the advantages of training on the larger, more diverse CADS-dataset over models
limited to the TotalSegmentator dataset. For the 119 mutual targets, the CADS-model achieves a mean
Dice score of 90.Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
The CADS-model is made accessible as a user-friendly plugin within 3D Slicer, integrated with SNOMED-CT terminology standards, aiming to advance robust AI solutions in radiology and provide comprehensive anatomical analysis for clinicians and researchers. p. 2 Furthermore, to facilitate practical adoption, we provide our model as a userfriendly plugin within 3D Slicer [38]. This tool is designed for both clinicians and researchers, offering
simple installation, one-click inference, and segmentation results presented according to the SNOMED-CT
terminology standard [39], with full access to the trained models and source code.Where the paper says it
Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.
Improvements for AI systems
-
Data-centric foundation models can be utilized to train models with enhanced generalization capabilities by utilizing a large volume of images, including those without manual annotations, which
enhance segmentation performance and model generalization compared to smaller, hand-annotated datasets alone.
-
Implement a multi-strategy approach for generating high-quality labels where the process involves three steps:
preparing distinct labeled data subsets,
training complementary models (“flavors”) on each subset,
andcombining their predictions to create final labels.
This ensures that the final label set integrates pseudolabels from the best-performing model flavor based on a performance-driven process. -
Develop a specialized refinement process for rib segmentation that includes using vesselFM to detect all tube-like structures throughout the body, effectively retrieving
the complete contour of ribs, intrinsically including the costovertebral joint regions.
This addressesfragmented or inconsistently labeled rib predictions
and missing costovertebral joints. -
Implement a two-step postprocessing strategy for refining head region pseudo-labels: first, applying a quality control filter by excluding scans with predicted brain voxel counts below 2,000 voxels, and second, performing targeted cropping to
remove FP predictions that extend into other body parts.
-
Modify the standard nnU-Net training process by performing oversampling that assigns sampling weights based on the inverse probability of class occurrence to mitigate the influence of class imbalance and ensure
balanced performance across all 167 target structures.
-
Implement a performance-driven process for selecting model flavors where each anatomical structure is evaluated using Dice Similarity Coefficient as the primary metric and 95-percentile Hausdorff distance as a secondary metric, ensuring that
the highest-ranked flavor’s pseudo-label becomes the final label for that structure in the CADS-dataset.
-
Enhance false negative penalization by setting Dice and NSD to 0 while assigning a maximum penalty value equal to the diagonal length of image bounding box for structures considered
truly missed
but still occupying a reasonably visible volume.
Abstract
Volumetric computed tomography (CT) segmentation supports quantitative image analysis, but most AI-based methods remain organ-specific, with fragmented models, inconsistent label definitions, and heterogeneous training and evaluation protocols. Here we present CADS, a unified data-model framework for scalable and generalizable whole-body CT segmentation. We construct a SNOMED-CT-referenced anatomical label space harmonizing heterogeneous cohorts from over 100 imaging centers, yielding complete annotations of 167 structures across 22,022 CT volumes and markedly expanding anatomical coverage and scale beyond existing resources. To complete labels on sparsely annotated CTs, we combine self-training with shape-informed quality control based on neural implicit functions, curating high-quality pseudo-labels. Trained on this resource, CADS shows robust whole-body segmentation across distribution shifts and improved performance relative to state-of-the-art approaches across 18 public datasets. We further validate it in an independent, expert-reviewed real-world oncology cohort of 2,864 CTs. The curated dataset, pretrained models, and user-friendly software together provide an accessible, reproducible foundation for large-scale anatomical segmentation and computational anatomy research.
Sources
- VISTA3D: A Unified Segmentation Foundation Model For 3D Medical Imaging
- Medical Vision Generalist: Unifying Medical Imaging Tasks in Context
- An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
- Towards Unifying Anatomy Segmentation: Automated Generation of a Full-body CT Dataset via Knowledge Aggregation and Anatomical Guidelines
- Generalist Foundation Models from a Multimodal Dataset for 3D Computed Tomography
- Vision Foundation Models for Computed Tomography
- STU-Net: Scalable and Transferable Medical Image Segmentation Models Empowered by Large-Scale Supervised Pre-training
- Computer Aided Detection for Pulmonary Embolism Challenge (CAD-PE)
- Automated Thoracolumbar Stump Rib Detection and Analysis in a Large CT Cohort
- nnU-Net Revisited: A Call for Rigorous Validation in 3D Medical Image Segmentation
- Data-Centric Foundation Models in Computational Healthcare: A Survey
- Rethinking Abdominal Organ Segmentation (RAOS) in the clinical scenario: A robustness evaluation benchmark with challenging cases
- vesselFM: A Foundation Model for Universal 3D Blood Vessel Segmentation
Related papers
- HDNet: High-resolution Dual-domain Learning for Spectral Compressive Imaging
- Mask-guided Spectral-wise Transformer for Efficient Hyperspectral Image Reconstruction
- FDIR: Fidelity-Anchored Decompressed Image Restoration for Human and Machine Perception
- A Disease-Centric Vision-Language Foundation Model for Precision Oncology in Kidney Cancer
- Task-Aware Bitrate Thresholds for Codec-Robust Ultrasound Segmentation
- RPL-UIE: Reference-Guided Prior Learning for Content-Preserving Underwater Image Enhancement