FGCARL: Fisher-Guided Continuously Adaptive Representation Learning for Medical Image Segmentation

arXiv:2611.16210 · eess.IV · Submitted 2026-11-12 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: Electrical Engineering. The day in electrical engineering: signals, systems and the hardware that carries them.

Rosa: We're looking at the paper: "FGCARL: Fisher-Guided Continuously Adaptive Representation Learning for Medical Image Segmentation".

Dev: Here's the gist of the paper. Deep learning (DL) has achieved remarkable success in medical image segmentation by learning data-driven representations from large-scale annotated datasets.

Rosa: FGCARL is proposed as a unified framework for domain- and class-incremental 3D volumetric medical image segmentation that simultaneously addresses evolving domains and newly introduced classes while retaining previously acquired knowledge.

Dev: Pruning redundant reasoning tokens The paper introduces FisherGuided Continuously Adaptive Representation Learning (FGCARL), a unified framework for domain- and class-incremental 3D volumetric medical image segmentation.

Rosa: This framework comprises two key components: Mutual Adaptive Instance Representation (MAIR) and Fisher-Guided Knowledge Retention (FGKR).

Paper summary: Rosa: So we're looking at this paper FGCARL which is about medical image segmentation using deep learning. Essentially, the idea is to create a unified framework that handles two big problems at once: when the data source changes—that’s the domain shift—and when you introduce new types of tissue or classes—that’s the class shift.

Dev: Right. What matters here is that this system has to adapt while still remembering what it learned before, so we aren't throwing everything away every time we see a new dataset or a new type of scan. The paper claims this FGCARL framework solves the challenge of evolving domains and newly introduced classes without forgetting old knowledge.

Taro: It sounds like the authors are tackling those two shifts simultaneously, which is tricky because usually, you either adapt to the domain or you expand the class set, not both at once. How do they manage that balance in their approach?

Rosa: They introduce two main parts: Mutual Adaptive Instance Representation and Fisher-Guided Knowledge Retention. MAIR seems to be handling the distribution alignment across different domains by using bidirectional normalization and denormalization to keep the features consistent between old and new data.

Dev: And then we have FGKR, which is supposed to stop catastrophic forgetting. It uses Fisher information to figure out which model parameters are really important for the tasks we've already learned, so it only constrains those specific parameters during adaptation.

Taro: So, if I'm a researcher looking at this from a real-world failure mode standpoint, how does this parameter-level constraint actually translate into graceful recovery when the world throws us something unexpected?

Rosa: Well, the paper says FGKR estimates the importance of model parameters to previously learned tasks using Fisher information. They approximate the increase in loss for old tasks as one over two times a sum over all parameters of that Fisher information multiplied by some squared difference between current and previously learned values.

Dev: That’s a specific way to quantify parameter importance, which is good because it’s not just guessing; it’s using information theory to guide the regularization. The overall training objective is set up so that you get plasticity for new knowledge acquisition, distribution alignment for domain consistency, and knowledge retention for old information.

Taro: If we look at how this works in practice, they are decoupling the segmentation task from the specific output space and sequence length dependencies. That suggests it might work with different types of networks without needing a complete rewrite of the architecture itself.

Rosa: Exactly. They designed it to work with both convolutional neural networks and vision transformers because MAIR operates directly on intermediate feature representations, which is pretty flexible for deployment across different setups.

Dev: It’s about making sure the features themselves are consistent across domains through that adaptive normalization and denormalization process, while also using higher-order moment matching for a non-parametric distribution alignment mechanism. That way they don't have to assume a specific mathematical shape for the data distributions they encounter.

Taro: What I’m curious about is the robustness when we look at the results. The paper shows that FGCARL maintains high segmentation performance across both previously learned and newly introduced targets under this unified setting. Does it really handle simultaneous domain and class shifts well in those tests?

Rosa: It does, qualitatively, they show that FGCARL preserves the overall morphology and spatial localization of the metastatic lesions even after adapting to a new domain. The tuning parameter lambda equals one point zero seems to give a good balance between keeping old knowledge and learning the new target.

Dev: That balancing act is key. If you increase that regularization weight from lambda equals zero, you generally see better segmentation performance, which shows each part of the framework contributes effectively to both continual adaptation and knowledge retention.

Taro: So for someone just listening to this, what does this mean? It means we can train a model on one set of scans and then it can keep learning new things or moving to different scanner types without having that old information completely wiped out.

Rosa: That’s the core idea, the implication is that we might not need to go through the whole process of re-training from scratch every time we get a new site or a new type of pathology.

Dev: It means for clinical applications, this could mean models can be deployed more flexibly across different hospitals or imaging centers without massive retraining cycles.

Taro: It also addresses the privacy issue because FGKR keeps historical knowledge by constraining parameters rather than storing the patient data itself, which is a big win for data security concerns.

Rosa: So, to wrap up on FGCARL: it's this unified solution for medical image segmentation continual learning that handles domain and class shifts while keeping old knowledge intact through parameter constraints instead of data rehearsal. That’s what the authors are proposing in "FGCARL: Fisher-Guided Continuously Adaptive Representation Learning for Medical Image Segmentation".

Conclusion: Rosa: So we're looking at FGCARL, which stands for Fisher-Guided Continuously Adaptive Representation Learning for Medical Image Segmentation. It’s by Rosa and Dev, and they’ve put together this framework to tackle some really tough problems in medical image segmentation.

Dev: Yeah, it seems like the core of what they did is unifying domain shift and class shift problems all at once. They are trying to make a model learn new things without forgetting everything it already knew.

Rosa: Right, and the big idea they’re pushing is that you can do this without having to constantly re-train the whole thing from scratch every time you get a new type of scan or a different tissue class.

Dev: The authors use two main components for this: Mutual Adaptive Instance Representation and Fisher-Guided Knowledge Retention. MAIR handles aligning the data features across different domains using bidirectional normalization, while FGKR selectively keeps the important parts of the model parameters that matter for old tasks.

Rosa: And what I find interesting is how they build their training signal to balance those things out at every stage of learning. They've got this total objective function with three parts: one for segmentation, one for distribution alignment, and one specifically for knowledge retention.

Dev: That means the model gets enough flexibility to learn new stuff, but it’s also being constrained by these two other parts to keep things stable and prevent catastrophic forgetting. It’s a balancing act between plasticity and stability.

Rosa: From where I'm sitting, this framework is designed to be architecture agnostic too. It works with both standard convolutional networks and vision transformers because the adaptation happens at the feature level, not tied to a specific network design.

Dev: That’s important for deployment; it means you don't need to rebuild your entire segmentation pipeline just because you switched from one type of network to another. But we gotta remember, how well this actually holds up when the world throws us something completely unexpected in a real clinical setting?

Rosa: Exactly. The paper shows that FGCARL maintains good segmentation performance even after adapting to new domains, and they’ve shown it works for both tumor regions they tested. It looks promising for making these models more robust in messy, real-world environments.

Xiaobing Yua, Jin Yangb, Peijie Qiua, Weiwei Maa, Xiaofeng Liuc

Mallinckrodt Institute of Radiology, Washington University School of Medicine in St. Louis, St. Louis, MO, USA · Department of Radiation Oncology, Icahn School of Medicine at Mount Sinai, New York, NY, USA · Departments of Radiology and Biomedical Imaging and Biomedical Informatics and Data Science at Yale University

eess.IV

Submitted: 2026-11-12

Updated: 2026-11-12

Comments: 19 pages, 5 figures

License: http://creativecommons.org/licenses/by/4.0/

Importance score: 83/100

The gist: Deep learning (DL) has achieved remarkable success in medical image segmentation by learning data-driven representations from large-scale annotated datasets.

Key concepts

Mutual Adaptive Instance Representation (MAIR)
MAIR adaptively aligns feature distributions across different imaging domains. It uses bidirectional normalization and denormalization to create consistent feature representations, allowing the model to handle new domains while maintaining statistical continuity with previously seen ones.
Fisher-Guided Knowledge Retention (FGKR)
FGKR preserves parameters crucial for past tasks by using Fisher information. This technique applies a regularization term that constrains parameter updates, ensuring that important knowledge from old domains and classes is retained without needing to store the original images or annotations.
Total Objective Function
The training objective combines three goals: segmentation accuracy (seg), distribution alignment (align), and knowledge retention (FGKR). This balanced approach allows the model to learn new information from new data while simultaneously ensuring it doesn't forget what it already learned.

Terminology

Summary

Deep learning (DL) has achieved remarkable success in medical image segmentation by learning data-driven representations from large-scale annotated datasets. p. 0

Where the paper says it

FGCARL: Fisher-Guided Continuously Adaptive Representation Learning for Medical Image Segmentation Xiaobing Yua,c,∗ , Jin Yangb,∗ , Peijie Qiua , Weiwei Maa and Xiaofeng Liuc aMallinckrodt Institute of Radiology, Washington University School of Medicine in St. Louis, 63110, MO, USA bDepartment of Radiation Oncology, Icahn School of Medicine at Mount Sinai, New York, 10029, NY, USA cDepartments of Radiology and Biomedical Imaging, and Biomedical Informatics and Data Science, Yale University, New Haven, 06519, CT, USA A R T I C L E I N F O Keywords: Incremental learning Domain Adaptation Medical Image Segmentation Representation learning A B S T R A C T Deep learning (DL) has achieved remarkable success in medical image segmentation by learning data-driven representations from large-scale annotated datasets.

FGCARL is proposed as a unified framework for domain- and class-incremental 3D volumetric medical image segmentation that simultaneously addresses evolving domains and newly introduced classes while retaining previously acquired knowledge. p. 0
Where the paper says it

This separation leaves a critical gap in developing a unified approach that can simultaneously adapt to evolving domains and newly introduced classes while retaining previously acquired knowledge. To address this challenge, we propose FisherGuided Continuously Adaptive Representation Learning (FGCARL), a unified framework for domain- and class-incremental 3D volumetric medical image segmentation.

Pruning redundant reasoning tokens

The paper introduces FisherGuided Continuously Adaptive Representation Learning (FGCARL), a unified framework for domain- and class-incremental 3D volumetric medical image segmentation. This framework comprises two key components: Mutual Adaptive Instance Representation (MAIR) and Fisher-Guided Knowledge Retention (FGKR). p. 0

Where the paper says it

To address this challenge, we propose FisherGuided Continuously Adaptive Representation Learning (FGCARL), a unified framework for domain- and class-incremental 3D volumetric medical image segmentation. FGCARL comprises two key components: Mutual Adaptive Instance Representation (MAIR), which adaptively aligns feature distributions across sequentially encountered domains through bidirectional normalization and denormalization, and Fisher-Guided Knowledge Retention (FGKR), which selectively preserves parameters important for previously learned domains and classes through Fisher-guided regularization.

MAIR adaptively aligns feature distributions across sequentially encountered domains through bidirectional normalization and denormalization. p. 0
Where the paper says it

To address this challenge, we propose FisherGuided Continuously Adaptive Representation Learning (FGCARL), a unified framework for domain- and class-incremental 3D volumetric medical image segmentation. FGCARL comprises two key components: Mutual Adaptive Instance Representation (MAIR), which adaptively aligns feature distributions across sequentially encountered domains through bidirectional normalization and denormalization, and Fisher-Guided Knowledge Retention (FGKR), which selectively preserves parameters important for previously learned domains and classes through Fisher-guided regularization.

FGKR selectively preserves parameters important for previously learned domains and classes through Fisher-guided regularization. p. 0
Where the paper says it

To address this challenge, we propose FisherGuided Continuously Adaptive Representation Learning (FGCARL), a unified framework for domain- and class-incremental 3D volumetric medical image segmentation. FGCARL comprises two key components: Mutual Adaptive Instance Representation (MAIR), which adaptively aligns feature distributions across sequentially encountered domains through bidirectional normalization and denormalization, and Fisher-Guided Knowledge Retention (FGKR), which selectively preserves parameters important for previously learned domains and classes through Fisher-guided regularization.

Extensive experiments demonstrate that FGCARL consistently outperforms state-of-the-art continual learning methods, effectively adapting to domain and class shifts while mitigating catastrophic forgetting. p. 0
Where the paper says it

We conducted extensive experiments across six cross-institutional segmentation tasks using diverse segmentation architectures. Experimental results demonstrate that FGCARL consistently outperformed state-of-the-art continual learning methods, effectively adapting to domain and class shifts while mitigating catastrophic forgetting and maintaining strong segmentation performance on both previously learned and newly encountered domains.

How the training signal is built

The overall objective function for FGCARL jointly optimizes segmentation, distribution alignment, and knowledge retention at each incremental stage. The total objective is defined as total = lambda seg seg + lambda dist align + lambda FG FGKR. p. 8

Where the paper says it

Unified Incremental Learning Objective Function FGCARL jointly optimizes segmentation, distribution alignment, and knowledge retention at each incremental stage. The overall objective is defined as  (𝑡+1) total = 𝜆seg (𝑡+1) seg + 𝜆dist (𝑡+1) align + 𝜆FG (𝑡+1) FGKR, (29) where  (𝑡+1) seg denotes the segmentation loss on the newly encountered domain and classes,  (𝑡+1) align encourages consistency between feature distributions across incremental domains via Wasserstein-2 distance, and  (𝑡+1) FGKR preserves parameters that are important for previously acquired knowledge.

The segmentation objective provides the plasticity required to acquire knowledge from newly introduced domains and anatomical classes. p. 8
Where the paper says it

The three terms play complementary roles. The segmentation objective provides the plasticity required to acquire knowledge from newly introduced domains and anatomical classes.

MAIR reduces domain-specific distributional discrepancies while maintaining statistical continuity with previously encountered domains. p. 8
Where the paper says it

The segmentation objective provides the plasticity required to acquire knowledge from newly introduced domains and anatomical classes. MAIR reduces domain-specific distributional discrepancies and maintains statistical continuity with previously encountered domains.

FGKR constrains updates to parameters important for previously acquired knowledge and thereby improves model stability. p. 8
Where the paper says it

MAIR reduces domain-specific distributional discrepancies and maintains statistical continuity with previously encountered domains. FGKR constrains updates to parameters that are important for previously acquired knowledge and thereby improves model stability.

Where the gains come from

The MAIR module performs bidirectional feature transformation through adaptive normalization and denormalization, enabling the model to establish consistent feature representations across previously encountered and newly observed domains. Forward MAIR normalizes the feature representation of the newly encountered domain and applies learnable affine transformations to retain the flexibility required for domain-specific adaptation. p. 4

Where the paper says it

MAIR consists of two complementary operations: Forward MAIR and Backward MAIR. Forward MAIR normalizes the feature representation of the newly encountered domain and applies learnable affine transformations to retain the flexibility required for domain-specific adaptation.

Backward MAIR subsequently reconstructs the representation using statistics associated with a previously learned domain, which enables rehearsal-free continual adaptation. p. 4
Where the paper says it

Forward MAIR normalizes the feature representation of the newly encountered domain and applies learnable affine transformations to retain the flexibility required for domain-specific adaptation. Backward MAIR subsequently reconstructs the representation using statistics associated with a previously learned domain.

Higher-order moment matching is used as a non-parametric distribution alignment mechanism to constrain the statistical characteristics of feature distributions without imposing an explicit parametric assumption. p. 7
Where the paper says it

The two alignment components within the framework play distinct but complementary roles. Higher-order moment matching characterizes and reduces discrepancies in the statistical shape of feature distributions without imposing an explicit parametric distributional assumption, making it suitable for capturing non-Gaussian characteristics of medical image representations.

Wasserstein alignment explicitly regularizes the residual discrepancy between the incoming and historically aligned representations, providing an explicit representation-level constraint during optimization. p. 7
Where the paper says it

Higher-order moment matching characterizes and reduces discrepancies in the statistical shape of feature distributions without imposing an explicit parametric distributional assumption, making it suitable for capturing non-Gaussian characteristics of medical image representations. In contrast, Wasserstein alignment explicitly regularizes the residual discrepancy between the incoming and historical embeddings, providing a direct consistency constraint during optimization.

Mitigation of catastrophic forgetting

FGKR addresses catastrophic forgetting by estimating the importance of model parameters to previously learned tasks using Fisher information. The estimated increase in the previous-domain loss is approximated as 1/2 sum j F(t) j (theta j - theta star j) squared. p. 8

Where the paper says it

(24) Using a diagonal approximation of the Fisher information matrix, the expected increase in the previous-domain loss can therefore be approximated as 𝐿 (𝑡) (𝜃̂) − 𝐿 (𝑡) (𝜃 ∗ ) ≈ 1 2 ∑ 𝑗 𝐹 (𝑡) 𝑗 ( 𝜃̂ 𝑗 − 𝜃 ∗ 𝑗 )2 . (25) The diagonal Fisher information for parameter 𝜃𝑗 can be estimated empirically as 𝐹̂ (𝑡) 𝑗 = 1 |(𝑡) | ∑ (𝑥,𝑦)∈(𝑡) [ 𝜕 log 𝑝(𝑦|𝑥; 𝜃 ∗ ) 𝜕𝜃𝑗 ]2 .

FGKR introduces a Fisher-weighted regularization term, such that parameters with large Fisher information receive stronger constraints and remain closer to their previously learned values. p. 8
Where the paper says it

(27) The gradient of this regularization term with respect to an individual parameter is 𝜕 (𝑡+1) FGKR 𝜕𝜃𝑗 = 𝐹̂ (𝑡) 𝑗 ( 𝜃𝑗 − 𝜃 ∗ 𝑗 ) . (28) Therefore, parameters with large Fisher information receive stronger constraints and remain closer to their previously learned values, whereas parameters with small Fisher information can undergo larger updates to accommodate newly encountered domains and classes.

This mechanism preserves historical knowledge through parameter-level constraints rather than storing previous images or annotations. p. 8
Where the paper says it

This allows historical knowledge to be retained throughout successive domain and class transitions without requiring replay of the corresponding historical patient data. Unlike rehearsal-based approaches, FGKR preserves historical knowledge through parameter-level constraints rather than storing previous images or annotations.

Robustness to simultaneous domain and class shifts

FGCARL is designed to simultaneously accommodate evolving imaging domains, including differences in modalities, acquisition protocols, and institutional distributions. The framework maintains high segmentation performance across both previously learned and newly introduced targets under the unified domain- and class-incremental setting. p. 0

Where the paper says it

This separation leaves a critical gap in developing a unified approach that can simultaneously adapt to evolving domains and newly introduced classes while retaining previously acquired knowledge. To address this challenge, we propose FisherGuided Continuously Adaptive Representation Learning (FGCARL), a unified framework for domain- and class-incremental 3D volumetric medical image segmentation.

Qualitative results show that FGCARL preserves the overall morphology and spatial localization of the metastatic lesions after adaptation to the new domain. p. 8
Where the paper says it

Unified Incremental Learning Objective Function FGCARL jointly optimizes segmentation, distribution alignment, and knowledge retention at each incremental stage. The overall objective is defined as  (𝑡+1) total = 𝜆seg (𝑡+1) seg + 𝜆dist (𝑡+1) align + 𝜆FG (𝑡+1) FGKR, (29) where  (𝑡+1) seg denotes the segmentation loss on the newly encountered domain and classes,  (𝑡+1) align encourages consistency between feature distributions across incremental domains via Wasserstein-2 distance, and  (𝑡+1) FGKR preserves parameters that are important for previously acquired knowledge.

Generalization across network architecture

The framework is compatible with both convolutional neural networks (CNNs) and vision transformers (ViTs) because MAIR operates directly on intermediate feature representations. The consistent improvements obtained with both nnU-Net and Swin UNETR support the architecture-agnostic design of FGCARL. p. 9

Where the paper says it

We employed Swin UNETR as the vision transformer (ViT)-based backbone and nnU-Net as the convolutional neural network (CNN)-based backbone [21, 26]. Both architectures were evaluated under the same incremental learning settings to assess the architecture-agnostic capability of FGCARL.

FGCARL can be directly integrated into different network architectures as an adaptive normalization mechanism, without requiring architecture-specific modifications. p. 2
Where the paper says it

• We introduce Mutual Adaptive Instance Representation, a feature-level adaptation module that performs bidirectional normalization and denormalization to align feature representations across sequentially encountered domains. MAIR enables effective adaptation to distribution shifts without requiring access to previously observed patient data and can be integrated with diverse segmentation architectures.

Parameter sensitivity analysis

The regularization terms exhibit similar trends across the evaluated tumor subregions, and lambda = 1.0 provides a favorable balance between retaining previously learned knowledge and adapting to the target domain. p. 4

Where the paper says it

Backward MAIR subsequently reconstructs the representation using statistics associated with a previously learned domain. In this way, MAIR provides a mechanism for adapting to new feature distributions while maintaining statistical consistency with previously acquired representations.

Increasing the regularization weight from lambda = 0 generally improves segmentation performance, indicating that each component contributes to effective continual adaptation and knowledge retention. p. 8
Where the paper says it

FGKR constrains updates to parameters that are important for previously acquired knowledge and thereby improves model stability. Through their joint optimization, FGCARL balances adaptation and retention, enabling a segmentation model to continuously learn from both evolving imaging domains and newly introduced anatomical classes without requiring rehearsal of historical patient data.

The all-layer configuration achieves the best performance across the three tumor regions and also obtains the highest DSC on both UCSF-BMSR and BrainMetShare. p. 0
Where the paper says it

Liu) ORCID(s): specific goals of the study, clinical needs, or newly available modalities [52, 38]. Specifically, as illustrated in Figure 1, a segmentation model was trained to segment enhancing tumor (ET) from three modalities, including pre-contrast T1 weighted (T1w-pre), post-contrast T1-weighted (T1w-post), and Fluid-Attenuated Inversion Recovery (FLAIR), in the UCSF-BMSR dataset [59].

Conclusion

FGCARL provides a unified solution to the three key challenges in medical image segmentation continual learning: simultaneous domain and class shifts, privacy constraints associated with rehearsal-based learning, and adaptation across heterogeneous segmentation architectures. The proposed framework enables a segmentation model to continuously learn from both evolving imaging domains and newly introduced anatomical classes without requiring rehearsal of historical patient data. p. 8

Where the paper says it

FGKR constrains updates to parameters that are important for previously acquired knowledge and thereby improves model stability. Through their joint optimization, FGCARL balances adaptation and retention, enabling a segmentation model to continuously learn from both evolving imaging domains and newly introduced anatomical classes without requiring rehearsal of historical patient data.

FGCARL demonstrates the effectiveness, robustness, and generalizability for domain- and class-incremental medical image segmentation. p. 2
Where the paper says it

By combining representationlevel domain adaptation with Fisher-guided parameter-level knowledge retention, FGCARL enables continual learning across domains with different imaging characteristics and evolving anatomical or pathological label spaces while mitigating catastrophic forgetting. Extensive experiments across multiple cross-institutional segmentation tasks and heterogeneous segmentation architectures demonstrate that FGCARL consistently outperforms state-of-the-art incremental learning methods, validating its effectiveness, robustness, and generalizability for domain- and class-incremental medical image segmentation.

Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.

Improvements for AI systems

  1. Unified framework for domain- and class-incremental learning: FGCARL enables continuous adaptation to evolving imaging domains (like different modalities) and expanding label spaces (new anatomical structures) simultaneously without requiring access to previously observed patient data.

  2. Mutual Adaptive Instance Representation (MAIR): This feature-level adaptation module performs bidirectional normalization and denormalization, aligning feature representations across sequentially encountered domains while allowing the model to establish consistent feature representations across previously encountered and newly observed domains.

  3. Fisher-Guided Knowledge Retention (FGKR): This rehearsal-free optimization strategy selectively preserves parameters important for previously learned tasks by using Fisher information, which mitigates catastrophic forgetting by constraining updates to parameters that carry critical historical knowledge.

  4. Non-Gaussian Distribution Alignment: The framework incorporates higher-order moment matching to align feature distributions, which progressively constrains the statistical characteristics of the feature distributions through multiple moments without imposing an explicit parametric distribution assumption.

  5. Embedding-Level Distribution Consistency: A Wasserstein-2 distance loss explicitly regularizes the residual discrepancy between incoming and historically aligned representations, ensuring that the incoming-domain embedding to remain close to the representation that has been realigned toward the previously learned domain.

  6. Architecture Agnostic Adaptation: FGCARL can be directly integrated into both Convolutional Neural Networks (CNNs) and Vision Transformers (ViTs), providing a unified framework for domain- and class-incremental 3D volumetric medical image segmentation regardless of the backbone used.

  7. Robustness to Domain Shifts: The combination of Forward and Backward MAIR allows the model to simultaneously accommodate distributional changes and maintain statistical continuity with previously learned domains, preventing knowledge from being overwritten by new domain statistics.

Abstract

Deep learning (DL) has achieved remarkable success in medical image segmentation by learning data-driven representations from large-scale annotated datasets. However, medical images are often collected independently across clinical sites, resulting in substantial domain shifts due to differences in imaging modalities, acquisition protocols, and intensity distributions. Meanwhile, segmentation classes may vary across sites due to differences in annotation standards, leading to class shifts. These evolving domains and class definitions can substantially degrade segmentation performance on newly encountered data, while sequential adaptation may cause catastrophic forgetting of knowledge acquired from previously encountered domains and classes. Existing incremental learning methods for medical image segmentation primarily address either domain shifts or class shifts, with these two settings largely investigated independently. This separation leaves a critical gap in developing a unified approach that can simultaneously adapt to evolving domains and newly introduced classes while retaining previously acquired knowledge. To address this challenge, we propose Fisher-Guided Continuously Adaptive Representation Learning, a unified framework for domain- and class-incremental 3D volumetric medical image segmentation. FGCARL comprises two key components: Mutual Adaptive Instance Representation (MAIR), which adaptively aligns feature distributions across sequentially encountered domains through bidirectional normalization and denormalization, and Fisher-Guided Knowledge Retention (FGKR), which selectively preserves parameters important for previously learned domains and classes through Fisher-guided regularization. Experimental results demonstrate that FGCARL consistently outperformed state-of-the-art methods, effectively adapting to domain and class shifts while mitigating catastrophic forgetting.

Sources

Related papers