CADS: A Comprehensive Anatomical Dataset and Segmentation for Whole-Body Anatomy in Computed Tomography

arXiv:2507.22953 · eess.IV, cs.CV · Submitted 2025-07-29 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: Electrical Engineering. The day in electrical engineering: signals, systems and the hardware that carries them.

Rosa: We're looking at the paper: "CADS: A Comprehensive Anatomical Dataset and Segmentation for Whole-Body Anatomy in Computed Tomography".

Dev: Here's the gist of the paper. Accurate delineation of anatomical structures in volumetric Computed Tomography (CT) scans is crucial for diagnosis and treatment planning, yet current approaches often suffer from a fragmented landscape of incompatible models with varying performance protocols.

Rosa: Foundational segmentation models designed to process multiple organs offer a holistic view, but robust clinical deployment requires comprehensive training data that addresses data heterogeneity and anatomical coverage—areas where existing whole-body approaches are deficient.

Dev: This work introduces CADS (Comprehensive Anatomical Dataset and Segmentation for Whole-Body Anatomy in Computed Tomography), an open-source framework prioritizing the systematic integration, standardization, and labeling of heterogeneous, cross-institutional, and cross-vendor CT data sources for whole-body CT segmentation.

Rosa: The core contribution is a large-scale dataset comprising 22,022 CT volumes with complete annotations for 167 anatomical structures.

Title and authors: Rosa: So we're looking at this paper today called "CADS: A Comprehensive Anatomical Dataset and Segmentation for Whole-Body Anatomy in Computed Tomography". It sounds like a big deal for getting AI to actually understand the whole body from CT scans.

Dev: Yeah, it's about building a dataset and a segmentation model that can handle all those structures across the entire body. The authors are Xu, Amiranashvili, Navarro, Fritsak, Hamamci, Shitmann, Wittmann, Christman-Christ3, de la Rosa1, Deseoe1, Graf8.

Rosa: What this means for us is that we're finally looking at a way to get comprehensive anatomical delineation instead of just looking at one organ in isolation. It moves us toward a holistic view.

Dev: Exactly. The authors are aiming to advance robust AI solutions in radiology and make comprehensive anatomical analysis accessible to clinicians and researchers alike, as mentioned on page two > <ref:2507.22953#pg1>

The paper's summary: Rosa: So what's the actual substance of this CADS paper? Basically, they're presenting an open-source framework. It’s not just a single model; it’s a whole system for organizing and labeling data from all over.

Dev: They are focusing on integrating heterogeneous, cross-institutional, and cross-vendor CT data sources systematically. The core contribution is this large dataset they put together.

Rosa: That dataset has twenty-two thousand twenty-two CT volumes with complete annotations for one hundred sixty-seven different anatomical structures <ref:2507.22953#pg3,22,022 CT volumes with>. That’s a huge scale compared to what’s out there right now.

Dev: It is eighteen times more scans than existing collections and sixty percent more distinct anatomical targets, which is a significant increase in coverage. They are using pseudo-labeling grounded in self-training through iterative model refinement, shape-guided qualification of those labels >

The paper's improvements: Rosa: The authors talk about how they built this system. It’s a multi-stage pipeline to go from raw data to the final segmentation model.

Dev: They started with training initial region-specific models on nine anatomical regions, which cover all those one hundred sixty-seven target structures, and then generating pseudo-labels across all twenty-two thousand twenty-two volumes > <ref:2507.22953#pg3>

Rosa: Then they have this quality control step where they use neural implicit functions and shape priors to filter out the unreliable pseudo-labels. That’s a smart way to clean up the noise.

Dev: And then they use a multi-strategy label optimization process. They combine three complementary models, or flavors, each trained with different data characteristics, to create the final CADS dataset >

Rosa: Finally, they train the main CADS model using customized class balancing strategies to make sure every structure gets attention. It’s a very detailed approach to building this comprehensive system.

Conclusion: Dev: So putting it all together, the paper presents the CADS framework, which includes both the dataset and a robust segmentation model suite trained on it. The whole point is accessibility and scale for whole-body analysis.

Rosa: It really shows how much more data and better organization we need to move beyond looking at just one organ at a time in CT scans. The performance metrics are pretty impressive across the board, with an overall Dice score of eighty-five point eight seven percent across all one hundred sixty-seven structures >

Dev: Especially for things like the heart, they got average Dice improvements of nine point two percent for myocardium and seven point five percent for pulmonary artery structures >

Rosa: It’s a lot of data diversity—spanning forty sources in sixteen countries—which means the model should generalize pretty well to different imaging protocols and patient groups.

Dev: But we have to remember what they admit about their process. They mention that closed-source models and deliverables hinder reproducibility, which is a real challenge for adoption >

Rosa: And they also pointed out accessibility barriers, like the absence of user-friendly tools, which can stop healthcare practitioners from actually integrating these solutions into clinical workflows >

Dev: So while the results on performance are strong across various regions—like the Liver getting a mean Dice score of ninety-six point nine four plus two point zero zero—the real hurdle is making sure everyone can use this tool reliably in a real hospital setting >

Department of Quantitative Biomedicine University of Zurich · ETH AI Center ETH Zurich · Department of Radiation Oncology University Hospital and University of Zurich · Faculty of Medicine University of Zurich · School of Computation Information and Technology Technical University of Munich · Department of Diagnostic and Interventional Neuroradiology School of Medicine and Health Technical University of Munich Departmental Imaging Institute in Artificial Intelligence in Medicine TUM School of Medicine and Health Institute for Radiation Medicine Helmholtz Zentrum München Departmental Imaging Institute in Artificial Intelligence in Medicine Klinikum rechts der Isar TUM School of Medicine and Health German Consortium for Translational Cancer Research Partner Site Munich University Hospital Essen Institute for Interventional and Diagnostic Radiology Pediatric Radiology and Neuroradiology University Medical Center Rostock Medical University of Vienna Department of Biomedical Imaging and Image-guided Therapy Medical University of Vienna Department of Radiology University Hospital Basel Departmental Imaging Institute in Artificial Intelligence in Medicine Essen Institute for Diagnostic and Interventional Radiology Pediatric Radiology and Neuroradiology University Medical Center Rostock Departmental Imaging Institute in Artificial Intelligence in Medicine Essen

eess.IV, cs.CV

Submitted: 2025-07-29

Updated: 2026-11-10

Journal ref: Nature Communications (2026)

DOI: 10.1038/s41467-026-78732-1

Code: https://github.com/murong-xu/CADS

License: http://creativecommons.org/licenses/by/4.0/

Importance score: 100/100

The gist: Accurate delineation of anatomical structures in volumetric Computed Tomography (CT) scans is crucial for diagnosis and treatment planning…

Key concepts

CADS Dataset
This is a massive collection of 22,022 CT volumes with complete labels for 167 different anatomical structures. It combines data from many different hospitals and sources to create a comprehensive resource for training AI models.
CADS-model
This is the fully automated segmentation model suite trained on the CADS dataset. It uses a 3D U-Net architecture and is designed to segment all 167 target structures consistently across various anatomical regions, like the heart or bones.
Multi-strategy Label Optimization
This process involves training several complementary models ('flavors') on different data subsets. The best model for each structure is then selected based on performance metrics, ensuring that the final dataset has high-quality labels for all structures.
Performance-driven Flavor Selection
When creating the final CADS dataset, each anatomical structure is evaluated using Dice Similarity Coefficient and Hausdorff distance. The label from the flavor that performs best for that specific structure is chosen as the definitive label.

Terminology

Summary

Accurate delineation of anatomical structures in volumetric Computed Tomography (CT) scans is crucial for diagnosis and treatment planning, yet current approaches often suffer from a fragmented landscape of incompatible models with varying performance protocols. p. 0

Where the paper says it

Accurate delineation of anatomical structures in volumetric Computed Tomography (CT) scans is crucial for diagnosis and treatment planning. While AI has advanced automated segmentation, current approaches typically target individual structures, creating a fragmented landscape of incompatible models with varying performance and disparate evaluation protocols.

Foundational segmentation models designed to process multiple organs offer a holistic view, but robust clinical deployment requires comprehensive training data that addresses data heterogeneity and anatomical coverage—areas where existing whole-body approaches are deficient. p. 0
Where the paper says it

Foundational segmentation models designed to process more organs address these limitations by providing a holistic anatomical view through a single model. Yet, robust clinical deployment demands comprehensive training data, which is lacking in existing whole-body approaches – both in terms of data heterogeneity and, more importantly, anatomical coverage.

This work introduces CADS (Comprehensive Anatomical Dataset and Segmentation for Whole-Body Anatomy in Computed Tomography), an open-source framework prioritizing the systematic integration, standardization, and labeling of heterogeneous, cross-institutional, and cross-vendor CT data sources for whole-body CT segmentation. p. 0

Where the paper says it

Yet, robust clinical deployment demands comprehensive training data, which is lacking in existing whole-body approaches – both in terms of data heterogeneity and, more importantly, anatomical coverage. In this work, rather than pursuing incremental optimizations in model architecture, we present CADS, an open-source framework that prioritizes the systematic integration, standardization, and labeling of heterogeneous cross-institutional and cross-vendor data sources for whole-body CT segmentation.

The core contribution is a large-scale dataset comprising 22,022 CT volumes with complete annotations for 167 anatomical structures. p. 0
Where the paper says it

In this work, rather than pursuing incremental optimizations in model architecture, we present CADS, an open-source framework that prioritizes the systematic integration, standardization, and labeling of heterogeneous cross-institutional and cross-vendor data sources for whole-body CT segmentation. At its core is a large-scale dataset of 22,022 CT volumes with complete annotations for 167 anatomical structures, representing a significant advancement in both scale and coverage, with 18 times more scans than existing collections and 60% more distinct anatomical targets.

This represents a significant advancement in scale and coverage, featuring 18 times more scans than existing collections and 60% more distinct anatomical targets. p. 0
Where the paper says it

In this work, rather than pursuing incremental optimizations in model architecture, we present CADS, an open-source framework that prioritizes the systematic integration, standardization, and labeling of heterogeneous cross-institutional and cross-vendor data sources for whole-body CT segmentation. At its core is a large-scale dataset of 22,022 CT volumes with complete annotations for 167 anatomical structures, representing a significant advancement in both scale and coverage, with 18 times more scans than existing collections and 60% more distinct anatomical targets.

The CADS framework consists of two main components: the CADS-dataset itself and the CADS-model, a robust, fully automated segmentation model suite trained on this dataset. p. 2

Where the paper says it

The collection spans more than 100 imaging facilities across 16 countries, capturing a wide spectrum of clinical variability. CADS-model: A robust, fully automated segmentation model suite trained on the CADS-dataset and thoroughly validated, capable of segmenting 167 structures from head to knee across diverse anatomical systems.

The development pipeline is multi-stage: (1) initial region-specific model training and pseudo-label generation across 22,022 volumes using specialized models for nine anatomical regions covering all 167 target structures; (2) automated quality control employing neural implicit functions and shape priors to filter unreliable pseudo-labels; (3) multi-strategy label optimization through an ensemble selection mechanism combining three complementary models (flavors) trained with varying data characteristics to create the assembled CADS-dataset; and (4) final CADS-model training with customized class balancing strategies.

The CADS-model utilizes a 3D full-resolution U-Net architecture with residual connections in the encoder path, selected for its robust performance and efficient runtime compatibility with nnU-Net's L configuration. The dataset is highly diverse, spanning 40 sources across 16 countries (including public challenges, TCIA collections, and newly acquired clinical data), encompassing normal and pathological cases across various imaging protocols. p. 4

Where the paper says it

Our data collection strategy systematically consolidates CT scans and annotations from three main sources: (1) large-scale public archives including TotalSegmentator [27] (1,203 annotated volumes with comprehensive coverage of 104 anatomical structures), which provides a foundational basis for whole-body segmentation, along with the National Lung Screening Trial (NLST [40], 7,172 scans), CT-RATE [41] (3,134 scans), and AbdomenCT-1K [42] (1,062 scans); (2) specialized annotation collections from medical imaging challenges (via Grand Challenge [36] and MICCAI [37]) and focused research studies (such as Han-Seg [43] for head-and-neck OARs, SAROS [44] for broad body context structures, and various oncology studies from TCIA [35]); and (3) newly contributed images and annotations, comprising manual annotations of critical OARs on the VISCERAL [12] dataset, 484 newly acquired anonymized head CTs, and 586 triphasic contrast-enhanced abdominal CT scans, which together strengthen coverage of cranial anatomy and multi-phase contrast imaging. This systematic integration unifies previously dispersed public datasets from medical imaging challenges and research initiatives, creating a unified resource that standardizes annotations while contributing new clinical data to the community.

The framework demonstrates superior performance over state-of-the-art approaches when evaluated across 18 public datasets and an independent real-world hospital cohort. p. 0

Where the paper says it

Through comprehensive evaluation across 18 public datasets and an independent real-world hospital cohort, we demonstrate advantages over state-of-the-art approaches. Notably, thorough testing of the model’s performance in segmentation tasks from radiation onarXiv:2507.

For cardiovascular structures, the CADS-model achieved significant average Dice improvements: myocardium (+9. p. 6
Where the paper says it

CADS 7 Analysis across major anatomical systems further underscores the importance of training on the larger and more comprehensive CADS-dataset. For cardiovascular structures, we observe significant average Dice improvements in myocardium (+9.

2%), pulmonary artery (+7. p. 3
Where the paper says it

4 Xu et al. Right ribs 1-12 Left ribs 1-12 Kidney R/L Upper lobe of lung R/L Middle lobe of lung R Lower lobe of lung R/L Adrenal gland R/L Inferior vena cava Portal vein and splenic vein Liver Aorta Spleen Gallbladder Stomach Pancreas Brain Face Colon Trachea Urinary bladder Duodenum Small intestine Esophagus Myocardium Heart atrium R/L Heart ventricle R/L Common iliac artery R/L Common iliac vein R/L Pulmonary artery Seminal vesicle Sigmoid colon Colostomy bag Mammary gland R/L Psoas major muscle R/L Rectus abdominis muscle R/L Heart Larynx Rectum Prostate Sternum Spinal canal Bones Muscle Spinal cord Pericardium Mediastinum Pituitary gland Cervical vertebrae C1-C7 Thoracic vertebrae T1-T12 Lumbar vertebrae L1-L5 Cochlear R/L Optic nerve R/L Carotid artery R/L Parotid gland R/L Arytenoid cartilage Cervical esophagus Lacrimal gland R/L Submandibular gland R/L Anterior eyeball segment R/L Posterior eyeball segment R/L Glottis Thyroid Mandible Pituitary gland Buccal mucosa Cricopharyngeus Both lips Oral cavity Brainstem Supraglottis Optic chiasm Thoracic cavity Abdominal cavity Gland structure Subcutaneous tissue Prosthetic breast implant White matter Spongy bone Compact bone Muscle of head Cerebrospinal fluid Scalp Blood Eyeball Gray matter Iliopsoas muscle R/L Gluteus minius muscle R/L Gluteus medius muscle R/L Gluteus maximus muscle R/L Deep muscle of back R/L Sacrum Hip R/L Femur R/L Clavicle R/L Scapula R/L Humerus R/L Fig.

5%), and heart chambers (+5. p. 6
Where the paper says it

2%), pulmonary artery (+7. 5%), and heart chambers (+5.

3–8.3%). Skeletal structures showed high precision, with ribs achieving 89. p. 6
Where the paper says it

5%), and heart chambers (+5. 3%) Skeletal structures demonstrate high precision in both sequential and individual elements: ribs achieve 89.

6–97.2% accuracy with mean improvements of +2. p. 6
Where the paper says it

3%) Skeletal structures demonstrate high precision in both sequential and individual elements: ribs achieve 89. 2% with improvements in 22/24 structures (mean +2.

9% across 22/24 structures, and vertebrae reaching 85. p. 1
Where the paper says it

Yet current resources face several fundamental constraints that limit the performance ceiling of models trained on them. These challenges include: Limited anatomical coverage – many approaches focus on specific regions or restricted structures, falling short of comprehensive whole-body analysis [31,32]; Insufficient data diversity – models developed on small-scale or single-source datasets show limited generalizability across institutions, populations, and imaging protocols [27,32,33,30,34]; Inconsistent annotation quality – existing automated labeling approaches introduce known systematic errors in complex anatomical structures like ribs and vertebrae [27], which models trained on such data learn and propagate; Human-dependent workflows – current approaches often rely on human-in-the-loop revisions during model training and dataset construction, limiting scalability and increasing operational costs [33,31]; Inadequate validation – evaluations are mostly conducted on internal or same-distribution datasets, with limited external testing, leaving real-world robustness unexplored [32,33,30,34]; Limited availability – some solutions remain

8–92.7% with mean improvements of +17/24 structures. p. 6
Where the paper says it

3%) Skeletal structures demonstrate high precision in both sequential and individual elements: ribs achieve 89. 2% with improvements in 22/24 structures (mean +2.

The CADS-model achieved an overall Dice score of 85. p. 6
Where the paper says it

The CADS-model extends beyond these mutual targets to segment 48 additional structures (shown by green cross markers in Figure 3), including critical sensory organs, body cavities, and reproductive structures. Across all 167 target structures, the CADS-model achieves an overall Dice score of 85.

87% across all 167 target structures, demonstrating consistent performance and robust generalization across diverse anatomical regions, including the Digestive System (e. p. 6
Where the paper says it

Complete performance metrics beyond Dice scores are provided in Online Method Section 5. Through systematic evaluation, the CADS-model trained on the CADS-dataset demonstrates consistent performance across diverse anatomical structures, with robust generalization across clinical settings.

g., Liver mean Dice score of 96. p. 5
Where the paper says it

Results demonstrate the advantages of training on the larger, more diverse CADS-dataset over models limited to the TotalSegmentator dataset. For the 119 mutual targets, the CADS-model achieves a mean Dice score of 90.

94 ± 2.00), Nervous System (e. p. 1
Where the paper says it

Unlike natural image datasets from RGB cameras that can contain millions of samples [23,24,25,26], medical datasets face two key limitations: they are orders of magnitude smaller and rarely annotated. Our analysis of 40 diverse sources reveals that only 21% of publicly available CT scans contain any annotations, with annotations typically confined to a single organ system (Table 1).

g., Brain mean Dice score of 97. p. 5
Where the paper says it

Results demonstrate the advantages of training on the larger, more diverse CADS-dataset over models limited to the TotalSegmentator dataset. For the 119 mutual targets, the CADS-model achieves a mean Dice score of 90.

29), Cardiovascular System (e. p. 1
Where the paper says it

Unlike natural image datasets from RGB cameras that can contain millions of samples [23,24,25,26], medical datasets face two key limitations: they are orders of magnitude smaller and rarely annotated. Our analysis of 40 diverse sources reveals that only 21% of publicly available CT scans contain any annotations, with annotations typically confined to a single organ system (Table 1).

g., Heart mean Dice score of 89. p. 5
Where the paper says it

Results demonstrate the advantages of training on the larger, more diverse CADS-dataset over models limited to the TotalSegmentator dataset. For the 119 mutual targets, the CADS-model achieves a mean Dice score of 90.

56), and Skeletal System (e. p. 1
Where the paper says it

Unlike natural image datasets from RGB cameras that can contain millions of samples [23,24,25,26], medical datasets face two key limitations: they are orders of magnitude smaller and rarely annotated. Our analysis of 40 diverse sources reveals that only 21% of publicly available CT scans contain any annotations, with annotations typically confined to a single organ system (Table 1).

g., Vertebra T10 mean Dice score of 84. p. 5
Where the paper says it

Results demonstrate the advantages of training on the larger, more diverse CADS-dataset over models limited to the TotalSegmentator dataset. For the 119 mutual targets, the CADS-model achieves a mean Dice score of 90.

72).

The CADS-model is made accessible as a user-friendly plugin within 3D Slicer, integrated with SNOMED-CT terminology standards, aiming to advance robust AI solutions in radiology and provide comprehensive anatomical analysis for clinicians and researchers. p. 2

Where the paper says it

Furthermore, to facilitate practical adoption, we provide our model as a userfriendly plugin within 3D Slicer [38]. This tool is designed for both clinicians and researchers, offering simple installation, one-click inference, and segmentation results presented according to the SNOMED-CT terminology standard [39], with full access to the trained models and source code.

Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.

Improvements for AI systems

  1. Data-centric foundation models can be utilized to train models with enhanced generalization capabilities by utilizing a large volume of images, including those without manual annotations, which enhance segmentation performance and model generalization compared to smaller, hand-annotated datasets alone.

  2. Implement a multi-strategy approach for generating high-quality labels where the process involves three steps: preparing distinct labeled data subsets, training complementary models (“flavors”) on each subset, and combining their predictions to create final labels. This ensures that the final label set integrates pseudolabels from the best-performing model flavor based on a performance-driven process.

  3. Develop a specialized refinement process for rib segmentation that includes using vesselFM to detect all tube-like structures throughout the body, effectively retrieving the complete contour of ribs, intrinsically including the costovertebral joint regions. This addresses fragmented or inconsistently labeled rib predictions and missing costovertebral joints.

  4. Implement a two-step postprocessing strategy for refining head region pseudo-labels: first, applying a quality control filter by excluding scans with predicted brain voxel counts below 2,000 voxels, and second, performing targeted cropping to remove FP predictions that extend into other body parts.

  5. Modify the standard nnU-Net training process by performing oversampling that assigns sampling weights based on the inverse probability of class occurrence to mitigate the influence of class imbalance and ensure balanced performance across all 167 target structures.

  6. Implement a performance-driven process for selecting model flavors where each anatomical structure is evaluated using Dice Similarity Coefficient as the primary metric and 95-percentile Hausdorff distance as a secondary metric, ensuring that the highest-ranked flavor’s pseudo-label becomes the final label for that structure in the CADS-dataset.

  7. Enhance false negative penalization by setting Dice and NSD to 0 while assigning a maximum penalty value equal to the diagonal length of image bounding box for structures considered truly missed but still occupying a reasonably visible volume.

Abstract

Volumetric computed tomography (CT) segmentation supports quantitative image analysis, but most AI-based methods remain organ-specific, with fragmented models, inconsistent label definitions, and heterogeneous training and evaluation protocols. Here we present CADS, a unified data-model framework for scalable and generalizable whole-body CT segmentation. We construct a SNOMED-CT-referenced anatomical label space harmonizing heterogeneous cohorts from over 100 imaging centers, yielding complete annotations of 167 structures across 22,022 CT volumes and markedly expanding anatomical coverage and scale beyond existing resources. To complete labels on sparsely annotated CTs, we combine self-training with shape-informed quality control based on neural implicit functions, curating high-quality pseudo-labels. Trained on this resource, CADS shows robust whole-body segmentation across distribution shifts and improved performance relative to state-of-the-art approaches across 18 public datasets. We further validate it in an independent, expert-reviewed real-world oncology cohort of 2,864 CTs. The curated dataset, pretrained models, and user-friendly software together provide an accessible, reproducible foundation for large-scale anatomical segmentation and computational anatomy research.

Sources

Related papers