XYScanNet: A State Space Model for Single Image Deblurring
Listen
Radio episode about this paper
Transcript
Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.
Tom: I'm Tom.
Jane: And I'm Jane, here with Lu, Meng and our in-house large language model, Lalam.
Tom: We're looking at the paper: "XYScanNet: A State Space Model for Single Image Deblurring".
Jane: Here's the gist of the paper. Deep state-space models (SSMs), like recent Mamba architectures, are emerging as a promising alternative to CNN and Transformer networks for vision tasks The proposed XYScanNet achieves competitive distortion metrics while markedly enhancing perceptual performance Addressing Spatial Misalignment The existing flatten-and-scan strategies in Mamba algorithms introduce two key challenges in a single scan First, short-range dependencies are lost due to the flattening of image patches into a 1D sequence, known as local pixel forgetting Second, distant pixels are incorrectly placed next to each other, disrupting spatial context, termed as spatial misalignment.
Tom: To solve this efficiently and preserve well-aligned spatial relationships, the paper introduces a straightforward slice-and-scan strategy that alternates scanning along intra- and inter-slices.
Jane: This design consists of interleaved Intra-Scanners and Inter-Scanners, each made up of horizontal and vertical branches.
Tom: The Vision State Space Module (VSSM) A new Vision State Space Module (VSSM) is designed for image deblurring to tackle the inefficiency challenges of current Mamba-based vision modules.
Paper summary: Jane: So, to wrap up on XYScanNet: A State Space Model for Single Image Deblurring, the authors are proposing this novel architecture using interleaved intra- and inter-VSSMs to solve the problems of local pixel forgetting and spatial misalignment in Mamba based vision systems >
Tom: They're also introducing the Vision State Space Module, or VSSM, which tackles inefficiency by having a dual-branch design for motion blur estimation at different angles >
Lu: The whole point is that this structure allows the model to recover both fine local details and estimate large-area blur across a wide range of scenarios >
Meng: For practical use, they show it cuts training time by fifty-six point sixty-one percent and inference memory usage by forty-six point eighteen percent compared to other approaches >
Jane: The paper suggests that this architecture achieves state of the art perceptual quality and competitive distortion metrics across diverse scenes, proving that SSMs are a viable alternative for vision tasks >
Tom: It’s about finding a better balance between the accuracy you need and the computational resources you have available when working with these state space models >
Conclusion: Tom: So we're wrapping up on XYScanNet, which is this State Space Model for Single Image Deblurring. Basically, they've built a new way to use these Mamba-like models to fix blurry pictures >
Jane: It’s about taking those powerful state space models and making them really good at deblurring images while keeping things efficient >
Tom: Right, so the authors are saying this XYScanNet architecture is able to do that, and they're claiming it gets competitive distortion metrics even while boosting how sharp the pictures look perceptually >
Lu: The core of what they did was dealing with those spatial problems—the local pixel forgetting and the spatial misalignment that usually happens when you flatten an image into a one-dimensional sequence >
Meng: That flattening is a real headache for training, because you lose the short-range connections and things get mixed up spatially >
Jane: And to fix that, they introduced this slice-and-scan strategy, alternating between scanning along the rows and scanning along the columns >
Tom: They also have this new Vision State Space Module, or VSSM. It uses two branches for motion blur estimation at different angles, which is a neat way to get a better picture of what's happening in the blur >
Lu: Then they layer on this dual gating feature fusion module called DGFF. It uses KL divergence terms between different feature levels to help the model decide which features are most important >
Jane: So, what’s the big deal here for us listening? Tom, you look at your training numbers—what does that efficiency gain actually mean when you're deploying this on a server? >
Tom: Well, they're talking about cutting training time by over fifty percent and memory usage by nearly forty-six percent compared to other methods >
Meng: Forty-six percent reduction in inference memory is significant for deployment because it means smaller models can run on less powerful hardware >
Jane: It seems like the main takeaway is that this architecture doesn't just get a better score on a benchmark; it actually finds a way to balance the quality of the fix with how much computational power you need >
Tom: Exactly. XYScanNet seems to show that these state space models can handle both fine details and big areas of blur at a very low computational cost compared to older vision methods >
Lu: It moves the needle on what we think is practical application for these recurrent architectures in restoration tasks >
Texas A&M University · Johns Hopkins University
cs.CV
Submitted: 2024-12-13
Updated: 2026-11-05
Comments: Accepted by CVPRW 2025. Marked all sub-figures with captions in Figure 5, which fixed the issue in the CVPR-W version
Journal ref: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) 2025, pp. 770-780
Code: https://github.com/HanzhouLiu/XYScanNet
License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
Importance score: 80/100
The gist: Deep state-space models (SSMs), like recent Mamba architectures, are emerging as a promising alternative to CNN and Transformer networks for vision tasks The proposed XYScanNet achieves competitive…
Key concepts
- Deep State Space Models (SSMs)
- These are advanced neural network architectures, like Mamba, that process data sequentially in a state-space manner rather than traditional CNNs or Transformers. They are promising for vision tasks because they can capture complex spatial dependencies efficiently while maintaining better performance than existing methods.
- Spatial Misalignment
- This occurs when flattening image patches into a 1D sequence causes two problems: short-range details are lost, and distant pixels end up incorrectly adjacent to each other. This disrupts the natural spatial context of the image, which XYScanNet aims to solve through its scanning strategy.
- Vision State Space Module (VSSM)
- This is a new component within XYScanNet designed specifically for deblurring. It uses a dual-branch design with horizontal and vertical components to estimate motion blur from different angles, allowing the model to better understand and correct the blur in the image.
Terminology
Summary
Deep state-space models (SSMs), like recent Mamba architectures, are emerging as a promising alternative to CNN and Transformer networks for vision tasks The proposed XYScanNet achieves competitive distortion metrics while markedly enhancing perceptual performance
Addressing Spatial Misalignment
The existing flatten-and-scan strategies in Mamba algorithms introduce two key challenges in a single scan First, short-range dependencies are lost due to the flattening of image patches into a 1D sequence, known as local pixel forgetting Second, distant pixels are incorrectly placed next to each other, disrupting spatial context, termed as spatial misalignment. To solve this efficiently and preserve well-aligned spatial relationships, the paper introduces a straightforward slice-and-scan strategy that alternates scanning along intra- and inter-slices. p. 0 This design consists of interleaved Intra-Scanners and Inter-Scanners, each made up of horizontal and vertical branches. p. 0
The Vision State Space Module (VSSM)
A new Vision State Space Module (VSSM) is designed for image deblurring to tackle the inefficiency challenges of current Mamba-based vision modules. The VSSM incorporates a dual-branch design, containing Scanner-Vertical (V) and-Horizontal (H) components to estimate motion blur at different angles. p. 3 The overall process of the VSSM is defined in Eq. p. 3 (3), which involves an element-wise multiplication and addition. p. 3
Feature Fusion Mechanisms
The paper introduces a lightweight feature fusion module called dual gating feature fusion (DGFF) to efficiently leverage multi-level features. DGFF introduces two feature-control gates, each consisting of a KL divergence term between the current-level and other-level features, followed by a sigmoid activation function. p. 3 This approach enhances model interpretability by measuring the distributional distance between feature levels. p. 3
Experimental Validation
The XYScanNet architecture is evaluated on multiple datasets, including GoPro [35], RWBI [57], and HIDE [43] test sets Experimental results show that XYScanNet enhances KID by 17% compared to the nearest competitor. Qualitative analysis highlights that the proposed Mamba-based method effectively recovers sharp edges and clear characters, producing images perceptually closer to the ground truth with noticeably enhanced visual quality. p. 4 Quantitative analysis in Table 1 shows that XYScanNet outperforms recent deblurring models across all perceptual metrics, achieving a KID score of 0. p. 4073.
Efficiency and Performance Gains
The interleaved approach contains K2 Intra-VSSMs and K2 InterVSSMs. This interleaved method improves PSNR on GoPro [35] by 0. p. 62 dB over the inter-only method, and demonstrates a 20. p. 639% reduction in training time and a 9. p. 652% decrease in memory usage compared to the intra-only approach. p. 6 Furthermore, the DGFF module reduces FLOPS by 49. p. 73% and Params by 53. p. 611%, demonstrating improved efficiency. p. 7 The proposed VSSM reaches a significant better balance between metric scores and computational costs, reducing training time by 56. p. 761% and inference memory usage by 46. p. 7 p. 718%.
Conclusion
The proposed XYScanNet, with interleaved Intra- and Inter-VSSMs, is capable of recovering both fine local details and estimating large-area blur. Experimental results on multiple synthetic and real-world datasets verify that XYScanNet achieves state-of-the-art perceptual quality and competitive distortion metrics across diverse scenes. p. 7 The work suggests that the proposed VSSM reaches a significant better balance between metric scores and computational costs, reducing the training time by 56. p. 761% and inference memory usage by 46.18%. The proposed XYScanNet, with interleaved Intra-and Inter-VSSMs, is capable of recovering both fine local details and estimating large-area blur. p. 7 The proposed XYScanNet, with interleaved Intraand Inter-VSSMs, is capable of recovering both fine local details and estimating large-area blur. p. 7 The proposed XYScanNet achieves state-of-the-art perceptual quality and competitive distortion metrics across diverse scenes. p. 7 p. 7 The proposed XYScanNet achieves state-of-the-art perceptual quality and competitive distortion metrics across diverse scenes.
Limitation
First, XYScanNet may not achieve an optimal balance between local and global blur estimation. Second, further discussions are needed to explore the relationship between loss functions and performance. p. 7 p. 7 Lastly, extending XYScanNet to other restoration tasks presents a promising direction. p. 7 p. 7 The limitation is that first, XYScanNet may not achieve an optimal balance between local and global blur estimation. p. 7 Second, further discussions are needed to explore the relationship between loss functions and performance. Lastly, extending XYScanNet to other restoration tasks presents a promising direction.
Acknowledgment
We appreciate the supports of Texas A&M High Performance Research Computing (HPRC) for providing access to the Grace and FASTER GPU clusters. The acknowledgment is that we appreciate the supports of Texas A&M High Performance Research Computing (HPRC) for providing access to the Grace and FASTER GPU clusters. p. 7
Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.
Improvements for AI systems
-
Slice-and-scan strategy to address spatial misalignment: The paper proposes a
slice-and-scan strategy that alternates scanning along intra- and inter-slices
topreserve wellaligned spatial relationships and solves spatial misalignment efficiently.
This allows the system to capture both local pixel dependencies through Intra-Scanners and global blur patterns across slices through Inter-Scanners. -
Vision State Space Module (VSSM) design: The VSSM is a novel module that
tackle[s] the inefficiency challenges of the current Mamba-based vision module
by being designed withinterleaved Intra- and Inter-Scanners,
leading to a significant reduction in training time by 56.61% and inference memory usage by 46.18% compared to MambaIR [17]. -
Cross-level feature fusion via Dual Gating Feature Fusion (DGFF): The system introduces DGFF, which
introduces two feature-control gates
using aKL divergence term between the current-level and other-level features,
enabling the model toselectively propagate visual information across multiple levels.
This enhances model interpretability by measuring the distributional distance between feature levels. -
Enhanced blur estimation capability: The architecture is capable of
estimating blur patterns with varying magnitudes and angles
because it utilizes a dual-branch design where Intra-Scanners capture pixel-level dependencies (horizontal and vertical) and Inter-Scanners capture cross-slice dependencies (vertical and horizontal). -
Improved perceptual performance on real data: Training on RealBlur datasets shows that XYScanNet
surpasses all recent models as rated by Q-ALIGN,
indicating acloser alignment with human perception in the real-world scenarios
and achieving an NIQE reduction of 0.163, validating enhanced naturalness.
Sources
- Demystifying MMD GANs
- Hungry Hungry Hippos: Towards Language Modeling with State Space Models
- Mamba: Linear-Time Sequence Modeling with Selective State Spaces
- Efficiently Modeling Long Sequences with Structured State Spaces
- MambaIR: A Simple Baseline for Image Restoration with State-Space Model
- Gaussian Error Linear Units (GELUs)
- Advanced Multiple Linear Regression Based Dark Channel Prior Applied on Dehazing Image and Generating Synthetic Haze
- DeblurDiNAT: A Compact Model with Exceptional Generalization and Visual Fidelity on Unseen Domains
- VMamba: Visual State Space Model
- Long Range Language Modeling via Gated State Spaces
- Mamba-360: Survey of State Space Models as Transformer Alternative for Long Sequence Modelling: Methods, Applications, and Challenges
- Simplified State Space Layers for Sequence Modeling
- Q-Align: Teaching LMMs for Visual Scoring via Discrete Text-Defined Levels
- Vision Mamba: Efficient Visual Representation Learning with Bidirectional State Space Model
- Barriers towards no-reference metrics application to compressed video quality analysis: on the example of no-reference metric NIQE
Related papers
- TEA: Text Encoder Alignment for Robust Concept Erasure in Text-to-Image Models
- Do Egocentric Video-Language Models Capture Both Hand- and Object-Centric Cues?
- Aura: Consistent Multi-Subject Video Generation via VLM-Grounded Semantic Alignment
- PVRF: All-in-one Adverse Weather Removal via Prior-modulated and Velocity-constrained Rectified Flow
- DLWM: Diverse Latent World Models for Efficient Multimodal Reasoning
- Out of Sight, Still in Mind: Token Compression for Omni-LLMs