Efficient Multi-Organ Segmentation Using SpatialConfiguration-Net with Low GPU Memory Requirements
Listen
Radio episode about this paper
Transcript
Introduction to the show: ident: Electrical Engineering. The day in electrical engineering: signals, systems and the hardware that carries them.
Rosa: We're looking at the paper: "Efficient Multi-Organ Segmentation Using SpatialConfiguration-Net with Low GPU Memory Requirements".
Dev: Here's the gist of the paper. Efficient Multi-Organ Segmentation Using SpatialConfiguration-Net with Low GPU Memory Requirements is a work that proposes an efficient multi-organ segmentation model based on SpatialConfiguration-Net (SCN) to improve generalization and reduce computational requirements for clinical use Even though many semantic segmentation methods exist that are able to perform well on many medical datasets, often, they are not designed for direct use in clinical practice This work employs a multi-organ segmentation model based on the SpatialConfiguration-Net (SCN), which integrates prior knowledge of the spatial configuration among the labelled organs to resolve spurious responses in the network outputs Furthermore, it modifies the architecture of the segmentation model to reduce its memory footprint as much as possible without drastically impacting the quality of the predictions Lastly, a minimal inference script is implemented for which both execution time and required GPU memory are optimized.
Rosa: The proposed approach involves a two-stage process where a localization model determines the region of interest (ROI) before the segmentation model based on SCN is employed The localization model, Mloc, is implemented as a coarse binary segmentation CNN that treats all labelled organs as the foreground label and is trained using the full image content after greatly downsampling to reduce memory footprint The segmentation model, Mseg, utilizes the ROI predicted by the localization model to reduce image memory by cropping it, allowing content within the ROI to retain a larger resolution compared to the image used by the localization model.
Dev: Model Architecture and Training The segmentation model Mseg is based on SCN, which was originally proposed for landmark localization and later adapted to multi-label segmentation of the whole heart The architecture of the localization model and the local network of the segmentation model follow a similar structure, consisting of a sequence of contracting blocks followed by expanding blocks, similar to U-Net The loss function for training is formulated as Lseg = LGD(yseg, ˆyseg) + λ l segLCE(yseg, ˆy l seg) + λ s segLCE(yseg, ˆy 2 seg), where LGD and LCE represent the generalized dice loss and cross-entropy loss functions respectively.
Rosa: The proposed segmentation model uses 1,270,090 parameters and requires 8,797,627,020 FLOPS for an input size of 32 × 32 × 32 and up to 879,762,672,300 FLOPS when using an input size of 160 × 128 × 160.
Paper summary: Rosa: So, we're looking at this paper called "Efficient Multi-Organ Segmentation Using SpatialConfiguration-Net with Low GPU Memory Requirements". Essentially, the main idea is that there are a lot of segmentation methods out there for medical images, but they often aren't set up for real clinical use because they struggle with new data or just take way too much computing power.
Dev: Right. And this work proposes a multi-organ segmentation model based on something called the SpatialConfiguration-Net, which they claim helps improve generalization to unseen data by using prior knowledge about where the organs are located in space.
Rosa: Exactly, and they also focus heavily on making it efficient. They modified the architecture of this network to shrink its memory footprint without significantly hurting how accurate the predictions are. Plus, they optimized the script for inference so it runs fast and uses less GPU memory when you actually try to use it in a clinical setting.
Taro: I'm interested in how they handle that spatial knowledge integration. They hypothesize that because this Spatial Configuration Network has both a local and a spatial network, the spatial part can integrate prior knowledge about the arrangement of organs which helps resolve those weird, spurious responses you see when the network gets confused by new images.
Dev: That's what they're aiming for with their design. They use a two-stage process where first, a localization model identifies the region of interest or ROI using all the image content at a low resolution, and then they use that ROI to crop out a high-resolution piece of the original image for their main segmentation model.
Rosa: So, to summarize what we've covered so far, this paper by Thaler et al. introduces the SpatialConfiguration-Net approach to multi-organ segmentation, claiming it offers better generalization by using spatial configuration knowledge and significantly reduces computational needs through architectural modifications and inference script optimization.
Dev: It really boils down to balancing that quality improvement with the need for efficiency in a practical medical environment where resources are limited. This sets up a good expectation for how this method might be applied outside of just lab testing.
Taro: When you look at the results they report, they see very strong performance for organs like liver and kidney, achieving average Dice Scores around ninety-five percent or higher on the validation set <ref:2111.13630#pg1>. But then there's a dip with the pancreas, which got an average Dice Score of eighty-two point three percent <ref:2111.13630#pg1>.
Rosa: That gap in performance across different organs is something they point out when they talk about their qualitative findings. They observed that even healthy organs can get mislabeled if there are very large lesions present in those individual organs.
Dev: So, the authors suggest that to solve those specific issues with large lesions, maybe they need to think about an imbalanced training scheme or a different data augmentation strategy where they artificially inject images with very large lesions into their training set.
Taro: It sounds like they're acknowledging that the model is good generally, but it needs more examples of those extreme cases before it can handle them reliably.
Rosa: And that brings us to the conclusion of this paper, "Efficient Multi-Organ Segmentation Using SpatialConfiguration-Net with Low GPU Memory Requirements". The core message is about making complex segmentation models usable in clinical practice by focusing on both spatial awareness and efficiency.
Dev: They demonstrate that they can achieve high Dice Scores, like ninety-five percent for the liver and kidney, while keeping the computational demands relatively low through their architectural adjustments <ref:2111.13630#pg1>. It's about getting good results without requiring massive GPU resources.
Rosa: The paper suggests that for organs like liver and kidney, performance is quite robust, but the limitations they acknowledge are related to data imbalance when dealing with very large lesions in those specific regions.
Taro: So, it’s not a complete fix for every organ or every scenario, but it's a solid step toward creating models that are more practical for real-world deployment where you can't just run these massive models on demand.
Dev: And the title itself speaks to the whole point: efficiency and low GPU memory requirements alongside multi-organ segmentation using SpatialConfiguration-Net. It’s a direct address to the practical constraints of deploying deep learning in healthcare settings.
Conclusion: Rosa: So we've been looking at this paper about efficient multi-organ segmentation using something called SpatialConfiguration-Net, and now we’re getting to what they actually concluded about it.
Dev: They’re talking about how they managed to pack spatial knowledge into the network without blowing up the memory requirements too much.
Rosa: Exactly. It’s not just another segmentation method; it's trying to make these complex models practical for real clinical use by being smart about how they organize their data internally.
Dev: They show good results on organs like liver and kidney, getting those high Dice Scores, but they also point out where the system struggles with things like very large lesions in the pancreas.
Rosa: That's a key part of it. It means that while the model is generally pretty strong for healthy structures, if you have something really big and messy in an organ, it can still get confused or mislabeled.
Dev: And they suggest that to fix those specific problems with large lesions, the training data needs to be different—maybe artificially adding more of those hard cases into the training set.
Rosa: So when you think about this for the real world, it means we’re getting tools that are lighter on compute and better at recognizing where things *should* be based on their arrangement in space.
Dev: It sets up a good baseline for what efficient multi-organ segmentation looks like today, but those limitations with large lesions are definitely the next thing to test out.
Gottfried Schatz Research Center: Biophysics Medical University of Graz Department of Computer Graphics and Vision Graz University of Technology Institute of Computer Graphics and Vision
eess.IV, cs.CV, cs.LG
Submitted: 2021-11-26
Updated: 2026-11-12
Code: https://github.com/franzthaler/FLARE21-EfficientSCN
License: http://creativecommons.org/licenses/by-sa/4.0/
Importance score: 80/100
The gist: Efficient Multi-Organ Segmentation Using SpatialConfiguration-Net with Low GPU Memory Requirements is a work that proposes an efficient multi-organ segmentation model based on…
Key concepts
- SpatialConfiguration-Net (SCN)
- This network integrates prior knowledge about how different organs are spatially arranged. It helps the model correct incorrect predictions by understanding the expected spatial relationships between labeled organs, leading to more accurate segmentation.
- Two-Stage Process
- The system uses two sequential models: first, a localization model (Mloc) identifies areas of interest (ROIs) by treating all organs as foreground labels on downsampled images. Second, the main segmentation model (Mseg) then crops these ROIs for higher-resolution processing, saving memory.
- Memory Optimization
- The architecture is modified to minimize the model's memory footprint without severely harming prediction quality. This is achieved through a two-stage approach and a minimal inference script, ensuring the system runs efficiently on devices with limited GPU resources.
- Multi-Organ Segmentation
- The model aims to segment multiple organs simultaneously, such as liver, kidney, and spleen. The SCN architecture helps resolve errors specific to these complex tasks by leveraging spatial context among the various labeled organs.
Terminology
Summary
Efficient Multi-Organ Segmentation Using SpatialConfiguration-Net with Low GPU Memory Requirements is a work that proposes an efficient multi-organ segmentation model based on SpatialConfiguration-Net (SCN) to improve generalization and reduce computational requirements for clinical use Even though many semantic segmentation methods exist that are able to perform well on many medical datasets, often, they are not designed for direct use in clinical practice This work employs a multi-organ segmentation model based on the SpatialConfiguration-Net (SCN), which integrates prior knowledge of the spatial configuration among the labelled organs to resolve spurious responses in the network outputs Furthermore, it modifies the architecture of the segmentation model to reduce its memory footprint as much as possible without drastically impacting the quality of the predictions Lastly, a minimal inference script is implemented for which both execution time and required GPU memory are optimized. p. 0 In this work, we employ a multi-organ
segmentation model based on the SpatialConfiguration-Net
(SCN), which integrates prior knowledge of the spatial configuration among the labelled organs to resolve spurious
responses in the network outputs. Furthermore, we modified the architecture of the segmentation model to reduce its
memory footprint as much as possible without drastically
impacting the quality of the predictions.Where the paper says it
Methodology Overview
The proposed approach involves a two-stage process where a localization model determines the region of interest (ROI) before the segmentation model based on SCN is employed The localization model, Mloc, is implemented as a coarse binary segmentation CNN that treats all labelled organs as the foreground label and is trained using the full image content after greatly downsampling to reduce memory footprint The segmentation model, Mseg, utilizes the ROI predicted by the localization model to reduce image memory by cropping it, allowing content within the ROI to retain a larger resolution compared to the image used by the localization model.
Model Architecture and Training
The segmentation model Mseg is based on SCN, which was originally proposed for landmark localization and later adapted to multi-label segmentation of the whole heart The architecture of the localization model and the local network of the segmentation model follow a similar structure, consisting of a sequence of contracting blocks followed by expanding blocks, similar to U-Net The loss function for training is formulated as Lseg = LGD(yseg, ˆyseg) + λ l segLCE(yseg, ˆy l seg) + λ s segLCE(yseg, ˆy 2 seg), where LGD and LCE represent the generalized dice loss and cross-entropy loss functions respectively. The proposed segmentation model uses 1,270,090 parameters and requires 8,797,627,020 FLOPS for an input size of 32 × 32 × 32 and up to 879,762,672,300 FLOPS when using an input size of 160 × 128 × 160. p. 2 The terms LGD and LCE represent the
generalized dice loss and cross-entropy loss functions respectively and λ
1
seg and λ
2
seg are hyperparameters used to
weight the individual loss functions. Overall, our segmentation model uses 1,270,090 parameters and requires
8,797,627,020 FLOPS for an input size of 32 × 32 × 32
and 879,762,672,300 FLOPS when using an input size of
160 × 128 × 160.Where the paper says it
Optimization for Efficiency
To optimize inference time and GPU memory, a minimal script is implemented where the same preprocessing strategy as during training is performed before generating predictions to compute the ROI The training protocols involved 4-fold cross-validation where each segmentation model was trained for 100,000 iterations for Mloc and 500,000 iterations for Mseg. The training times observed were Mloc at 11. p. 3 Training protocols. Data augmentation
methods
Rotation, translation, scaling, elastic
deformation, intensity shift and
scaling
Initialization of the
network He normal initialization
Batch size 1
Patch sampling
strategy Mloc
full image
Patch size Mloc [32 × 32 × 32] to [80 × 80 × 256]
Image spacing Mloc 6 × 6 × 6 mm
Total iterations Mloc 1,000,000
Patch sampling
strategy Mseg
ROI from ground truth (training) or
localization model (inference)
Patch size Mseg [32 × 32 × 32] to [160 × 128 × 160]
Image spacing Mseg 2 × 2 × 2 mm
Total iterations Mseg 500,000
Optimizer Adam (β1 = 0. Training time Mloc 11. 0 hours
Training time Mseg 49.Where the paper says it
Where the paper says it
Performance Evaluation
Quantitative results from the 4-fold cross-validation show good performance, with the average Dice Score (DSC) for liver, kidney, and spleen being 97.8%, 95.0%, and 97.2% respectively. p. 2 Defining the input and output of the local network
as x
1
seg and ˆy
1
seg and of the spatial network as x
2
seg and ˆy
2
seg
allows formulate the loss function as:
Lseg = LGD(yseg, ˆyseg) + λ
1
segLCE(yseg, ˆy
1
seg)
+ λ
2
segLCE(yseg, ˆy
2
seg),
(2)
where the ground truth yseg was resampled identically to
the image xseg. The terms LGD and LCE represent the
generalized dice loss and cross-entropy loss functions respectively and λ
1
seg and λ
2
seg are hyperparameters used to
weight the individual loss functions. The average Dice Score
(DSC) of liver, kidney and spleen are 97. 2%, respectively, while the pancreas has the lowest DSC
of 82. The
quantitative results of the submitted model on the validation
set given in Table 5 follow the same trend with liver, kidney
and spleen performing a lot better in terms of DSC and NSD
on average compared to the pancreas. The segmentation of
both, liver and spleen, achieved a DSC of over 90% closely
followed by the kidney with 89. In contrast, the validation set contains images from 9
different centers of which 8 have been unseen during training as well as images with various phases, including many
samples with phases also unseen during training. Consequently, while the cross-validation set can be used to evaluate how well the model performs on the task, the evaluation
on the validation set shows how well the model generalizes
to data from different centers with various phases.Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Qualitative Findings
Qualitative results illustrate that while healthy organs are segmented very well, large lesions in individual organs can lead to incorrect labeling or mislabeling by the segmentation model For instance, difficult cases from the validation set showed large lesions in the liver and spleen that were not labeled correctly. This observation suggests that additional data containing very large lesions would be beneficial to solve problems observed in these regions. p. 6 In contrast, large lesions of individual organs are only present in a small number of images which can lead to lacking segmentation predictions
for unseen images if this type of pathology was never or
not sufficiently encountered during training. This indicates,
that additional data containing very large lesions would be
beneficial to solve the problems observed in these regions. This indicates,
that additional data containing very large lesions would be
beneficial to solve the problems observed in these regions. Other potential solutions, which are not based on acquiring additional data, are to either use an imbalanced training
scheme, where images with large lesions are selected more
frequently during training, or a more sophisticated data augmentation strategy, where large lesions are injected artificially into the image using, e.Where the paper says it
Where the paper says it
Conclusion
The segmentation for liver, kidney, and spleen performs very well with each achieving a DSC of at least 95%, while the segmentation of the pancreas only reached 82.3% The results on the validation set confirm that liver and spleen achieved a DSC over 90% closely followed by the kidney with 89. p. 4 The
quantitative results of the submitted model on the validation
set given in Table 5 follow the same trend with liver, kidney
and spleen performing a lot better in terms of DSC and NSD
on average compared to the pancreas. The segmentation of
both, liver and spleen, achieved a DSC of over 90% closely
followed by the kidney with 89. The segmentation of
both, liver and spleen, achieved a DSC of over 90% closely
followed by the kidney with 89. 74%, however, the pancreas
only managed to reach a DSC of 75. This gap in performance is caused by the
data used to train and evaluate the respective model, as the
cross-validation models were trained and evaluated only on
subsets of the training set, while the submitted model was
trained on the full training set and evaluated on the validation set. As can be seen in Table 1, the training set only
consists of images obtained from two centers with the same
phase.Where the paper says it
Where the paper says it
Where the paper says it
Acknowledgment
The authors of this paper declare that the segmentation method they implemented for participation in the FLARE challenge has not used any pre-trained models nor additional datasets other than those provided by the organizers. This work was supported by the Austrian Research Promotion Agency (FFG): 871262. p. 6 Acknowledgment
The authors of this paper declare that the segmentation
method they implemented for participation in the FLARE
challenge has not used any pre-trained models nor additional datasets other than those provided by the organizers. This work was supported by the Austrian Research Promotion Agency (FFG): 871262.Where the paper says it
Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.
Improvements for AI systems
-
Spatial Configuration Network integration resolves spurious responses in network outputs by integrating
prior knowledge of the spatial configuration among the labelled organs.
This allows for more robust segmentation predictions on unseen data acquired from different scanners. -
The localization model Mloc, adapted from U-Net, determines a Region of Interest (ROI) to reduce memory footprint before employing the SCN segmentation model. This two-stage approach improves efficiency and accuracy by focusing the computationally intensive segmentation on relevant areas.
-
The proposed loss function Lseg combines generalized dice loss with cross-entropy loss between the ground truth and intermediate predictions from both the local and spatial networks. This multi-loss approach ensures that both subnetworks contribute to a plausible final prediction, as formulated in Equation 2 of the paper.
-
Inference time and required GPU memory are optimized through a minimal inference script that performs preprocessing identically to training but minimizes computational overhead. This optimization allows for deployment on resource-constrained clinical systems while maintaining prediction quality.
Abstract
Even though many semantic segmentation methods exist that are able to perform well on many medical datasets, they are often not designed for direct use in clinical practice. The two main concerns are generalization to unseen data with a different visual appearance, e.g., images acquired using a different scanner, and efficiency in terms of computation time and required Graphics Processing Unit (GPU) memory. In this work, we employ a multi-organ segmentation model based on the SpatialConfiguration-Net (SCN), which integrates prior knowledge of the spatial configuration among the labelled organs to resolve spurious responses in the network outputs. Furthermore, we modified the architecture of the segmentation model to reduce its memory footprint as much as possible without drastically impacting the quality of the predictions. Lastly, we implemented a minimal inference script for which we optimized both execution time and required GPU memory.
Sources
- A large annotated medical image dataset for the development and evaluation of segmentation algorithms
- The Liver Tumor Segmentation Benchmark (LiTS)
- Improving neural networks by preventing co-adaptation of feature detectors
Related papers
- HDNet: High-resolution Dual-domain Learning for Spectral Compressive Imaging
- Mask-guided Spectral-wise Transformer for Efficient Hyperspectral Image Reconstruction
- FDIR: Fidelity-Anchored Decompressed Image Restoration for Human and Machine Perception
- A Disease-Centric Vision-Language Foundation Model for Precision Oncology in Kidney Cancer
- CADS: A Comprehensive Anatomical Dataset and Segmentation for Whole-Body Anatomy in Computed Tomography
- Task-Aware Bitrate Thresholds for Codec-Robust Ultrasound Segmentation