UNet with Self-Adaptive Mamba-Like Attention and Causal-Resonance Learning for Medical Image Segmentation
summary
The gist
Medical image segmentation plays an important role in various clinical applications, but existing deep learning models face trade-offs between efficiency and accuracy.
In short
SAMA-UNet is a new U-shaped architecture for medical image segmentation that combines a Self-Adaptive Mamba-Like Attention block and a Causal-Resonance Multi-Scale Module. This combination improves accuracy on MRI, CT, and endoscopy by efficiently capturing both fine local details and broad global context, outperforming other models.
Key concepts
- SAMA Block
- This block replaces the standard Mamba SSM with a softmax attention mechanism. It uses contextual self-attention and dynamic weight modulation to efficiently process features, balancing high representation capacity with reduced computational complexity for better anatomical pattern recognition.
- CR-MSM
- This module enhances feature fusion by explicitly preserving the natural order of information flow across different scales. It uses directional multi-view transformations combined with SSM to ensure that fine details at higher resolutions correctly support the formation of coarser semantic features in a structured way.
- U-shaped Architecture
- SAMA-UNet uses a novel U-shaped structure designed to improve efficiency and accuracy. This design allows the model to effectively capture both local, high-resolution feature extraction (via the SAMA block) and global context aggregation (via CR-MSM), leading to superior segmentation results.
- Causal Relationships
- This refers to the inherent sequential or hierarchical order of information in medical images. The CR-MSM leverages these relationships by ensuring that low-level features are processed before high-level ones, which maintains semantic consistency between the encoder and decoder during feature fusion.
Terminology used across episodes
This episode discusses
- UNet with Self-Adaptive Mamba-Like Attention and Causal-Resonance Learning for Medical Image Segmentation · Paper Radio
- An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
- Mamba: Linear-Time Sequence Modeling with Selective State Spaces
- Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Duality
- FER-YOLO-Mamba: Facial Expression Detection and Classification Based on Selective State Space
- Demystify Mamba in Vision: A Linear Attention Perspective
- MambaOut: Do We Really Need Mamba for Vision?
- LongNet: Scaling Transformers to 1,000,000,000 Tokens
- Deformable DETR: Deformable Transformers for End-to-End Object Detection
- A Survey on Mamba Architecture for Vision Applications
- U-Mamba: Enhancing Long-range Dependency for Biomedical Image Segmentation
- VM-UNet: Vision Mamba UNet for Medical Image Segmentation
- Differential Transformer
- 2017 Robotic Instrument Segmentation Challenge
The paper
UNet with Self-Adaptive Mamba-Like Attention and Causal-Resonance Learning for Medical Image Segmentation · Read on arXiv
Saqib Qamar, Mohd Fazil, Parvez Ahmad, Shakir Khan, Abu Taha Zamani
Faculty of Computing and IT (FCIT), Sohar University · Department of Intelligent Systems, KTH Royal Institute of Technology, Stockholm, Sweden · College of Computer and Information Sciences, Imam Mohammad Ibn Saud Islamic University (IMSIU) · Auckland Bioengineering Institute, University of Auckland · Department of Computer Science, Faculty of Science, Northern Border University
Medical image segmentation plays an important role in various clinical applications; however, existing deep learning models face trade-offs between efficiency and accuracy. Convolutional Neural Networks (CNNs) capture local details well but miss the global context, whereas transformers handle the global context but at a high computational cost. Recently, State Space Sequence Models (SSMs) have shown potential for capturing long-range dependencies with linear complexity; however, their direct use in medical image segmentation remains limited due to incompatibility with image structures and autoregressive assumptions. To overcome these challenges, we propose SAMA-UNet, a novel U-shaped architecture that introduces two key innovations. First, the Self-Adaptive Mamba-like Aggregated Attention (SAMA) block adaptively integrates local and global features through dynamic attention weighting, enabling an efficient representation of complex anatomical patterns. Second, the causal resonance multi-scale module (CR-MSM) improves encoder-decoder interactions by adjusting feature resolution and causal dependencies across scales, enhancing the semantic alignment between low- and high-level features. Extensive experiments on MRI, CT, and endoscopy datasets demonstrate that SAMA-UNet consistently outperforms CNN, Transformer, and Mamba-based methods. It achieves 85.38% DSC and 87.82% NSD on BTCV, 92.16% and 96.54% on ACDC, 67.14% and 68.70% on EndoVis17, and 84.06% and 88.47% on ATLAS23, establishing new benchmarks across modalities. These results confirm the effectiveness of SAMA-UNet in combining efficiency and accuracy, making it a promising solution for real-world clinical segmentation tasks. The source code is available on GitHub.
DOI: 10.1038/s41598-025-28885-8
Transcript
Introduction to the show: ident: Electrical Engineering. The day in electrical engineering: signals, systems and the hardware that carries them.
Rosa: We're looking at the paper: "UNet with Self-Adaptive Mamba-Like Attention and Causal-Resonance Learning for Medical Image Segmentation".
Dev: Here's the gist of the paper. Medical image segmentation plays an important role in various clinical applications, but existing deep learning models face trade-offs between efficiency and accuracy.
Rosa: SAMA-UNet is a novel U-shaped architecture that introduces two key innovations to overcome the challenges of existing methods.
Dev: Extensive experiments on MRI, CT, and endoscopy datasets demonstrate that SAMA-UNet consistently outperforms CNN-, Transformer-, and Mamba-based methods.
Rosa: Architectural Innovations The proposed SAMA-UNet architecture is designed to enhance the efficiency and accuracy of medical image segmentation by incorporating several key innovations.
Title and authors: Rosa: So we're looking at this paper called "UNet with Self-Adaptive Mamba-Like Attention and Causal-Resonance Learning for Medical Image Segmentation". It’s written by Saqib Qamar, Mohd Fazil, Parvez Ahmad, Shakir Khan, and Abu Taha Zamani from Sohar University and KTH Royal Institute of Technology.
Dev: That title tells us a lot about what they are trying to do here. It combines three big ideas: Mamba-like attention, self-adaptive elements in the attention mechanism, and causal resonance learning for medical image segmentation. It sounds like they are trying to fix the trade-off between speed and getting an accurate look at medical images.
Taro: I wonder what that means for us when we're looking at these models. Are we talking about a model that can handle the massive scale of data without taking forever to process it?
Rosa: Exactly. It suggests they’ve found a way to make the model more efficient than standard Transformer models, which are great for global context but computationally heavy, and also better than traditional CNNs, which miss those big picture relationships.
Dev: They claim this architecture consistently outperforms CNNs, Transformers, and Mamba-based methods across MRI, CT, and endoscopy datasets. They show results like eighty-five point three eight percent DSC on BTCV and ninety-six point five four percent NSD on ACDC > <ref:2505.15234#pg2>. That’s a solid performance boost when you look at the metrics they report > <ref:2505.15234#pg2>.
Taro: But I want to know how this efficiency plays out in the real world, Rosa. When things go wrong, does this model handle unexpected noise better than what we're used to?
Rosa: That’s where the next part of their idea comes in. They introduce a SAMA block which replaces the SSM in the Mamba module with a softmax attention mechanism that has linear computational complexity > <ref:2505.15234#pg1>. This means it should be faster than what you'd expect from a standard Mamba structure, while still keeping that contextual awareness > <ref:2505.15234#pg1>.
Dev: Linear complexity is key for deployment, Dev. We’re talking about something manageable in terms of latency and memory usage when we try to run these things in clinical settings > <ref:2505.15234#pg1>. But how do they make sure that this attention mechanism isn't just a cheap way to process the data without capturing the right details?
Taro: That brings us to the CR-MSM module, which they call Causal-Resonance Multi-Scale Module. They say this module enhances multi-scale feature fusion by explicitly preserving the natural order of information flow across scales > <ref:2505.15234#pg3>.
Title and authors: Rosa: What does that mean in practical terms for an image segmentation task? It sounds like they are forcing the model to learn details at higher resolutions first before it builds up the coarser semantic features down below > <ref:2505.15234#pg3>.
Dev: They specifically address a limitation where traditional fusion strategies mix high- and low-level features without caring about their hierarchy, which messes up the semantic consistency between the encoder and decoder > <ref:2505.15234#pg3>. They use directional multi-view transformations combined with SSM to capture spatial dependencies from multiple perspectives > <ref:2505.15234#pg3>.
Taro: So if we think about how a human looks at an image, this sounds like prioritizing the fine details first, like focusing on a small spot and then getting the bigger picture > <ref:2505.15234#pg3>. I wonder if that mirrors any real-world sensory processing?
Rosa: They mention they also introduce a novel scan-expand method to enhance the 2D-Selective-Scan operation within this module > <ref:2505.15234#pg1>. That sounds like they’re making sure the scanning process is smarter, not just blindly moving across the image > <ref:2505.15234#pg1>.
Dev: The results are impressive, though there are always caveats with these kinds of experiments. They achieved eighty-four point zero six percent and eighty-eight point four seven percent on ATLAS23 and EndoVis17, for example > <ref:2505.15234#pg2>. That’s a significant gain over existing methods across those different modalities > <ref:2505.15234#pg2>.
Taro: But what about the limitations they themselves pointed out? They acknowledge that integrating the Mamba macro-architecture into the SAMA module increases the parameter count > <ref:2505.15234#pg3>. That’s a real cost when you're trying to optimize for deployment > <ref:2505.15234#pg3>.
Rosa: And they also mentioned that extending this architecture to three-dimensional medical image segmentation requires substantially higher memory because it triples the number of scanning sequences in the Vision State Space module > <ref:2505.15234#pg3>. That’s a big hurdle for real-time three dee applications > <ref:2505.15234#pg3>.
Dev: So, to summarize what we have so far, this paper presents the UNet with Self-Adaptive Mamba-Like Attention and Causal-Resonance Learning for Medical Image Segmentation, which uses a SAMA block for efficient attention and a CR-MSM to manage multi-scale fusion better > <ref:2505.15234#pg1>.
Taro: It sounds like the core idea is balancing that global context with the fine local detail, but making it happen in a way that respects the natural flow of information across scales > <ref:2505.15234#pg3>.
Rosa: Right. And it’s doing this better than what we see from standard CNNs or even other Mamba implementations, especially when you test it on varied datasets like MRI and CT > <ref:2505.15234#pg2>.
Dev: The operational side is still important though. They report that SAMA-UNet has 29 point 03M parameters and one hundred forty-five point six seven GFLOPs, which they say shows a balanced trade-off for clinical systems with limited hardware resources > <ref:2505.15234#pg2>.
Title and authors: Taro: If we look at what this means for the field right now, it seems like they are pushing toward models that can be powerful without demanding massive computational overhead from the start > <ref:2505.15234#pg1>.
Rosa: Yeah, and for clinical application, that efficiency is a big deal because it makes it possible to integrate these kinds of high-accuracy tools into systems that might not have access to supercomputers > <ref:2505.15234#pg2>.
Dev: So we’re seeing a model that achieves high accuracy on challenging medical images while trying to keep the parameter count and FLOPs in check, which is exactly what controls engineers are focused on > <ref:2505.15234#pg2>.
Taro: What I’m thinking is that the causal resonance idea might be a way to enforce a more sensible structure on how features interact during the segmentation process > <ref:2505.15234#pg3>.
Rosa: It sounds like they are really digging into how information moves through the network, making sure that when we look at low-level details, it actually helps build up those high-level semantic understandings > <ref:2505.15234#pg3>.
Dev: We need to keep an eye on those scaling issues for three dee work though, as they flag the memory requirements there as a major constraint > <ref:2505.15234#pg3>. That’s the part where we need to build out some more memory-efficient processing strategies > <ref:2505.15234#pg3>.
Taro: I think the real future work here needs to focus on those lightweight variants and model compression techniques to make it truly ready for real-time deployment in clinical workflows > <ref:2505.15234#pg1>.
Rosa: It seems like this UNet with Self-Adaptive Mamba-Like Attention and Causal-Resonance Learning for Medical Image Segmentation is putting a lot of effort into marrying the strengths of different architectures to get better performance without just making things bigger > <ref:2505.15234#pg2>.
Dev: So, to wrap up on this paper, it’s a step forward by showing how attention mechanisms can be made adaptive and how we can control the feature flow through causal relationships for better segmentation accuracy > <ref:2505.15234#pg1>.
Taro: It’s definitely a model that shows promise for combining efficiency with accuracy in real-world clinical segmentation tasks across MRI, CT, and endoscopy > <ref:2505.15234#pg2>.
Rosa: That’s what it is. We'll keep an eye on how they tackle those three dee memory issues and see if they can make it more accessible for broader use > <ref:2505.15234#pg3>.
Dev: Alright, that covers our look at this paper, UNet with Self-Adaptive Mamba-Like Attention and Causal-Resonance Learning for Medical Image Segmentation.
The paper's summary: Rosa: So, to recap, we’re looking at this UNet architecture that uses Mamba-like attention and causal resonance learning specifically for medical imaging segmentation on MRI, CT scans, and endoscopy images.
Dev: Yeah. The core idea is trying to fix that old trade-off between getting high accuracy and keeping the computational load low enough for actual clinical use.
Rosa: Exactly. They put in this SAMA block which uses a softmax attention mechanism with linear complexity instead of what’s standard, which should make it run much faster on those deep networks.
Dev: That's the efficiency angle we care about most. And then they have the CR-MSM module, that Causal-Resonance Multi-Scale Module, which is supposed to fix how different resolutions of data talk to each other in the network.
Rosa: They’re essentially forcing the model to respect the natural order of information flow, so fine details at one scale actually help build up the coarser features at another scale.
Dev: That sounds like they’re trying to make sure when you look at a small spot in a scan, that detail is used correctly to define what the bigger area is.
Rosa: It means better semantic alignment between the parts of the network that see fine details and the parts that see the overall context.
Dev: And they claim this setup consistently beats CNNs, Transformers, and other Mamba-based methods on four different medical datasets. They're talking about gains in metrics like DSC and NSD on things like BTCV data.
Rosa: The numbers are pretty solid, showing that this combination of attention adaptation and causal fusion actually delivers better diagnostic precision across MRI, CT, and endoscopic data than what we currently have.
Dev: But we gotta look at the trade-offs they mentioned too. They admit that the SAMA block has a parameter count, around twenty-nine million parameters, which is not zero.
Rosa: Right. And they also flag that if you want to go three-dimensional for full three dee segmentation, things get much heavier because the scanning sequences in their Vision State Space module triple.
Dev: So the authors are telling us this architecture is a step forward for accuracy and efficiency on 2D tasks, but scaling it up to real three dee medical scans is still going to require some serious memory optimization <ref:2505.15234#pg1>.
Rosa: Right. It’s a very promising structure for getting that global context without the massive computational overhead of older models.
Dev: So the next thing we need to watch is how they address those memory constraints when moving from two dimensions to three dimensions in their future work.
The paper's improvements: Tom: So, we're talking about the specific improvements they propose for this SAMA-UNet architecture on page six of their paper. Rosa, what’s their main suggestion?
Rosa: They suggest integrating a pixel-focused attention mechanism inspired by human vision where the fovea prioritizes high resolution detail in the center while processing more broadly around it.
Dev: So it’s like giving the model a way to look super close at a specific area without losing sight of the whole picture.
Rosa: Exactly. It dynamically allocates attention weights so that fine local details get more importance while still capturing that global context, which is what we talked about earlier.
Dev: That sounds like they're trying to solve the problem where standard attention mechanisms just treat everything equally, regardless of importance or location in the image.
Rosa: They also have this differential attention mechanism which cancels out noise tokens so the AI focuses more effectively on exactly what it needs to see, without needing extra parameters.
Dev: That’s a way to improve performance without adding any new computational cost or parameters, which is good for keeping things efficient.
Rosa: And they also have this Causal-Resonance Multi-Scale Module, CR-MSM, which is supposed to adjust the feature resolution and the causal dependencies across scales.
Dev: Remember how we said it was about forcing the model to respect the natural order of information flow? This is how it does that by making sure low level details actually come first before they build up into semantic understanding.
Rosa: Right. It’s like making sure you learn the texture of the surface before you understand what kind of object that surface belongs to.
Dev: The implication here is a more structured way for the network to learn features hierarchically, which should lead to more consistent results when segmenting complex anatomy.
Rosa: It means this isn't just a faster model; it’s supposed to be a smarter one because the way it processes information is fundamentally reorganized.
Dev: So they are focusing on making the attention mechanism itself adapt based on what the image is showing, rather than just using fixed weights across all inputs.
Rosa: That’s right. It moves away from a one-size-fits-all approach to feature weighting in segmentation tasks across all those different medical modalities.
Dev: If this works as they claim, it means we could see a real improvement in diagnostic confidence when using these kinds of AI tools for things like CT scans or endoscopies.
Conclusion: Tom: So we're wrapping up this look at "UNet with Self-Adaptive Mamba-Like Attention and Causal-Resonance Learning for Medical Image Segmentation". Rosa, what’s the final word on what this architecture actually does?
Rosa: It shows that combining selfadaptive attention with causal multi-scale fusion lets the AI capture both fine local details and broad global context much better than existing methods.
Dev: That’s the main takeaway for me. It moves beyond just chasing high scores on a benchmark dataset and shows a real structural improvement in how the network learns from an image.
Rosa: Exactly. It’s about making the network smarter by controlling how it pays attention to different parts of the input data during segmentation tasks across MRI, CT, and endoscopy images.
Dev: And from an engineering standpoint, it’s promising because they managed to keep the computational overhead relatively balanced with a parameter count around twenty-nine million parameters.
Rosa: It makes high-accuracy segmentation more accessible for clinical systems that might not have access to massive supercomputers for training or inference.
Dev: But we still have those scaling issues, Rosa, especially when they try to push this architecture into three dimensions, where the memory requirements triple because of the Vision State Space module.
Rosa: Right. So while it’s great for 2D work now, that three-dimensional application still needs serious memory-efficient processing strategies before it can be widely used in that domain <ref:2505.15234#pg1>.
Dev: And we need to keep an eye on those future variants they mention—the ones that focus on making it lighter and more suitable for real-time clinical workflows.
Taro: I just want to push back a little on the limitations. The authors mentioned the Mamba macro-architecture integration increases the parameter count, which is a definite cost when you’re trying to optimize for deployment latency.
Rosa: That’s true, Taro, but they argue that this trade-off is worth it because the resulting performance boost is significant across those diverse datasets.
Dev: I agree with Rosa on that. The safety case here relies on the fact that the precision gain justifies the added complexity in terms of computational resources.
Taro: For me, what matters is how graceful a system recovers when things misbehave in a real-world trial setting, so we need to see how this architecture handles unexpected input gracefully.
Rosa: Well, for now, these results on "UNet with Self-Adaptive Mamba-Like Attention and Causal-Resonance Learning for Medical Image Segmentation" show a strong combination of efficiency and accuracy in 2D medical imaging <ref:2505.15234#pg1,UNet with Self-Adaptive Mamba-Like Attention and Causal-Resonance Learning for>.
Dev: It’s definitely a step forward by showing how attention mechanisms can be made adaptive and how we can control the feature flow through causal relationships to get better segmentation results.
Taro: I think we need to keep watching those three dee memory issues closely because that’s where the real autonomy test will come for this kind of model.
More episodes
- Daily Summary for 2026-11-19
- 2111.07910-Mask-guided Spectral-wise Transformer for Efficient Hyperspectral Image Reconstruction
- 2203.02149-HDNet: High-resolution Dual-domain Learning for Spectral Compressive Imaging
- 2205.10102-Degradation-Aware Unfolding Half-Shuffle Transformer for Spectral Compressive Imaging
- 2311.10959-Structure-Aware Sparse-View X-ray 3D Reconstruction
- 2403.04116-Radiative Gaussian Splatting for Efficient X-ray Novel View Synthesis
- 2410.02572-Combining Pre- and Post-Demosaicking Noise Removal for RAW Video
- 2604.17453-Learned Nonlocal Feature Matching and Filtering for RAW Image Denoising
- 2603.20387-End-to-End Multi-Task Learning for Adjustable Joint Noise Reduction and Hearing Loss Compensation
- 2507.20825-Chirp-Permuted AFDM: A Versatile and Multifunctional Waveform for 6G