HazeSpikeMamba: Coupling Spiking-Inspired and State-Space Features for Self-Supervised Real-World Dehazing

summary

The gist

HazeSpikeMamba proposes HazeSpikeMamba, a compact dehazing framework that combines a spiking-inspired local path and an attentive state-space global path in a multiscale U-Net This work addresses the…

In short

HazeSpikeMamba is a compact dehazing framework using a five-stage U-Net that merges two paths: a local spiking path (TPCNNSpike) for fine details and a global state-space path (MambaBlock) for scene distribution. It is trained on synthetic data and then adapted to real haze using self-supervised reconstruction, showing consistent performance improvements across multiple image quality metrics.

Key concepts

TPCNNSpike
This local path is inspired by Pulse-Coupled Neural Networks (PCNN). It updates neurons in parallel by using the previous firing states of their neighbors, weighted by a Gaussian distribution. This allows the model to capture fine, local spatial details crucial for accurate image restoration.
MambaBlock
This global path uses an Attentive State-Space Module adapted from MambaIRv2. It models scene-wide haze distributions using linear complexity state-space modeling, grouping tokens based on semantic prompts to efficiently capture long-range dependencies across the entire image.
DegModel Adaptation
The model is adapted to real images by freezing a degradation reconstruction network (DegModel). This allows the model to learn a self-supervised signal by mapping its dehazed predictions back into the hazy domain, enabling fine-tuning on unlabeled data without needing paired ground truth.
Dual-Branch Fusion
Feature fusion occurs at each U-Net stage via a scalar gate. This gate merges the outputs from the local TPCNNSpike branch and the global MambaBlock branch, ensuring that both local spatial information and global scene context are effectively combined for better feature representation.

Terminology used across episodes

This episode discusses

The paper

HazeSpikeMamba: Coupling Spiking-Inspired and State-Space Features for Self-Supervised Real-World Dehazing · Read on arXiv

School of Artificial Intelligence and Electronic Engineering, Sichuan Technology and Business University · Chengdu University of Technology, College of Nuclear Technology and Automation Engineering · Sigray, Inc.

Dehazing networks are commonly trained on synthetic hazy-clear pairs, but their performance often drops on real photographs. Synthetic haze generated using the atmospheric scattering model does not fully capture the variability of real haze, and paired real hazy-clear images are scarce. In this work, we propose HazeSpikeMamba, a compact dehazing framework that combines a spiking-inspired local path and an attentive state-space global path in a multi-scale U-Net. The local path uses TPCNNSpike, a new spike-emission scheme inspired by the neighborhood coupling of Pulse-Coupled Neural Network (PCNN). Unlike grouped directional scanning, TPCNNSpike updates all neurons in parallel using the previous firing states of their Gaussian-weighted neighborhoods. The global path adapts the Attentive State-Space Module of MambaIRv2, retaining semantic prompting and sequence reordering while removing the window self-attention branch. Its state-space processing models long-range dependencies with complexity linear in sequence length. For target-domain adaptation, a frozen degradation network, pretrained on paired NH-HAZE data, re-synthesizes haze from the dehazed prediction. The reconstruction error updates only the final restoration layers of HazeSpikeMamba without haze-free labels during adaptation. A shared checkpoint is adapted once on each complete unlabeled target set, making the evaluation dataset-level and transductive rather than zero-shot or per-image optimization. The forward network contains 2.02M active parameters and requires 13.27G nominal MACs (measured with thop at 256x256 input). This adaptation consistently improves BRISQUE and NIMA on RTTS, URHI, and HSTS. On RTTS, BRISQUE decreases from 30.13 to 27.72 and NIMA increases from 4.13 to 4.87. Under this transductive protocol, the adapted model also achieves the best BRISQUE and NIMA on URHI and HSTS among the compared methods.

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: We're looking at the paper: "HazeSpikeMamba: Coupling Spiking-Inspired and State-Space Features for Self-Supervised Real-World Dehazing".

Jane: Here's the gist of the paper. HazeSpikeMamba proposes HazeSpikeMamba, a compact dehazing framework that combines a spiking-inspired local path and an attentive state-space global path in a multiscale U-Net This work addresses the challenge of real-world image dehazing by combining TPCNNSpike with an MHSA-free MambaBlock to capture both local details and scene-wide haze distributions Model Architecture and Components The HazeSpikeMamba framework is implemented as a five-stage U-Net that utilizes a dual-branch MambaSpikeBlock at each stage, where the local branch applies a stack of TPCNNBlocks and the global branch applies a stack of MambaBlocks The local path uses TPCNNSpike, which is inspired by the neighborhood coupling of Pulse-Coupled Neural Network (PCNN) to update all neurons in parallel using previous firing states of their Gaussian-weighted neighborhoods The global path adapts the Attentive State-Space Module of MambaIRv2, retaining semantic prompting and sequence reordering while removing the window self-attention branch Feature fusion between the local and global paths is achieved by a stage-level scalar gate which merges the two branch outputs after each dual-branch stack Training Strategy for Real-World Adaptation The model is first trained on paired synthetic data, and then it is adapted to unlabeled real hazy images using a frozen degradation reconstruction network The adaptation scheme involves a frozen DegModel that maps the dehazed prediction back to the observed hazy domain, providing a self-supervised reconstruction signal for updating only the final restoration layers of HazeSpikeMamba This adaptation is performed once on each complete unlabeled target set, making it dataset-level and transductive rather than zero-shot or perimage optimization The training objective for the fine-tuning stage is the unweighted sum of a reconstruction term and a Learned Perceptual Image Patch Similarity (LPIPS) term Contribution of Specialized Pathways The two feature paths serve complementary functions in haze removal, as detailed in the discussion section The TPCNNSpike activation acts as an iterative spatial communication process where each unit receives firing information from a Gaussian-weighted neighborhood over successive iterations The MambaBlock models scene-wide haze distributions through linear-complexity state-space modeling with ASE and SGN, which groups tokens assigned to the same prompt category to shorten sequence distance Component ablation studies show that TPCNNBlock performs better than MambaBlock on both perceptual metrics, suggesting local neighborhood interactions are fundamental for image restoration Performance and Adaptation Results The adapted model consistently improves BRISQUE and NIMA on RTTS, URHI, and HSTS after the self-supervised fine-tuning protocol For instance, on RTTS, BRISQUE decreases from 30.13 to 27.72 and NIMA increases from 4.13 to 4.87 The best results are highlighted in bold, while the second-best results are underlined, showing that adaptation improves every reported metric across all three datasets DegModel Training Source Influence The source data used to learn the degradation model significantly impacts adaptation performance When evaluating the effect of the DegModel training source on RTTS adaptation, NH-HAZE yields the strongest result on both perceptual metrics This suggests that physically generated, spatially nonhomogeneous haze in NH-HAZE provides a more informative relationship between scene content and degradation Furthermore, including a broader set of scenes in the degradation-learning data improves the adapted result compared to restricting the source to images resembling the outdoor target domain HazeSpikeMamba provides a compact approach to real-image dehazing by combining TPCNNSpike with an MHSA-free attentive state-space module in a multi-scale U-Net The framework is compact, requiring 2.02M active parameters and 13.27G MACs for the forward network The method is dataset-level and transductive, improving BRISQUE and NIMA on RTTS, URHI, and HSTS after adaptation The ablations support the complementary contributions of TPCNNBlock and MambaBlock NH-HAZE gives the best adapted result among the three DegModel training sources evaluated on RTTS The conclusions remain limited by the transductive protocol, reliance on no-reference quality metrics, and the small size of HSTS Future work should compare TPCNNSpike with grouped-scan activations under equal model capacity The paper is 2608.

Paper summary: Jane: So to wrap up this discussion on HazeSpikeMamba, the authors are proposing this specific combination of spiking-inspired local processing and an attentive state-space global path within a multi-scale U-Net.

Tom: It’s a compact framework that uses two point zero two million active parameters, and they showed it can improve perceptual metrics like BRISQUE and NIMA on RTTS, URHI, and HSTS after this self-supervised fine-tuning process.

Lu: The core idea is combining the TPCNNSpike local path with the MambaBlock global path through stage-level feature fusion to capture both local details and scene-wide haze distributions efficiently.

Meng: What does this mean for us in terms of practical deployment? It’s a relatively small model size, so it might be feasible for real-time applications where we need quick restoration.

Jane: The implication is that you don't necessarily need massive models to achieve good performance on real-world hazy images if you design your architecture to handle both the local and global information in this coupled way.

Tom: They also showed that the adaptation process is dataset-level and transductive, meaning it works well on a whole target set at once, which is much more practical than having to optimize per individual image.

Lu: The paper shows that the complementary roles of TPCNNBlock and MambaBlock are justified; local neighborhood interactions are fundamental for the restoration part.

Jane: And they also highlighted a key limitation in their own work: the entire adaptation process relies on this transductive protocol, which means we haven't tested how robust it is if we move to a zero-shot setting.

Meng: That’s fair; relying on that transductive setup is a constraint you have to keep in mind when planning real production pipelines.

Conclusion: Tom: So we've been looking at HazeSpikeMamba, which is this compact framework they put together using spiking neural networks and state-space models for image dehazing.

Jane: It really is a compact approach because the authors designed it as a multi-scale U-Net that mixes local detail processing with global scene understanding.

Tom: And the architecture itself is interesting—it uses this dual branch structure where one side handles local spatial communication and the other handles scene-wide distributions.

Lu: What's compelling is how they combine the TPCNNSpike, which models neighborhood interactions, with MambaBlocks that use state-space modeling to capture those large-scale haze patterns.

Meng: From a deployment standpoint, it’s good because they keep the parameter count relatively low—two point zero two million active parameters—so it doesn't immediately blow up your GPU memory.

Jane: But the real story is how they adapt this model to actual real-world hazy photos using a self-supervised fine-tuning method based on a degradation reconstruction network.

Tom: That adaptation step is crucial because it means the model learns how to fix haze based on what it sees in images, rather than just being trained on perfect synthetic data.

Lu: They showed that the local path, the TPCNNSpike part, actually performs better than the global MambaBlock part when we look at perceptual metrics like BRISQUE and NIMA.

Jane: So what does this mean for us? It suggests that for real-world image dehazing, you don't have to rely on one giant attention mechanism or one monolithic model; a hybrid approach works well.

Tom: Exactly. The results show concrete improvements across three different test datasets—RTTS, URHI, and HSTS—after this fine-tuning process.

Meng: I’m still skeptical about the adaptation protocol itself because it’s transductive, meaning it only works on a set of target images you already have in front of you.

Jane: That's a fair point about the limitation, but for now, the fact that this hybrid model consistently beats previous methods on those metrics is worth watching.

Tom: So to recap, HazeSpikeMamba couples spiking-inspired local processing with state-space global modeling in a U-Net structure to handle real-world haze, and adaptation improves performance on multiple image quality scores.

Lu: The next big thing will be figuring out if you can swap out the TPCNNSpike for other types of spatial communication methods while keeping the MambaBlock for the global part. (Sound of light static fade in)

More episodes

← Home