Efficient All-in-One Weather Restoration using Spectral Harmonization
Listen
Radio episode about this paper
Transcript
Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.
Tom: We're looking at the paper: "Efficient All-in-One Weather Restoration using Spectral Harmonization".
Jane: Here's the gist of the paper. Efficient All-in-One Weather Restoration using Spectral Harmonization presents FReSH-IR, a novel lightweight all-in-one restoration method that explicitly decomposes feature representations into high- and low-frequency components at each scale of a hierarchical encoder-decoder architecture <ref:2609.02839#pg2> By combining spectral decomposition with spatial processing through Fourierbased skip connections, FReSH-IR captures complementary frequency information without sacrificing spatial detail <ref:2609.02839#pg5> Our approach achieves similar restoration quality with 80% fewer parameters and operations than transformerbased models <ref:2609.02839#pg2> Extensive experiments demonstrate that our method offers a great efficiency-performance tradeoff, highlighting its practical applications in constrainedresource systems <ref:2609.02839#pg2> The proposed FReSH-IR follows a U-shaped CNN-based backbone <ref:2609.02839#pg2> Given a corrupted image I ∈ R 3×H×W, a 3×3 convolutional layer first projects the input into shallow feature embeddings of size C × H × W <ref:2609.02839#pg2> These embeddings are processed through a hierarchical encoder that progressively downsamples the features while explicitly decomposing them into low- and high-frequency components at each scale <ref:2609.02839#pg2> The low-frequency component is refined through a stack of Residual Blocks, and both components are recombined before downsampling <ref:2609.02839#pg2> At the bottleneck, a Dual-Attention Module (DAM) captures long-range spatial dependencies <ref:2609.02839#pg2> A carefully designed decoder progressively recovers the features, recombining the upsampled features with the corresponding encoder features <ref:2609.02839#pg2> Each block processes the high and low components separately, only fusing them during the last block of each level <ref:2609.02839#pg2> Frequency-Aware Feature Extraction At each encoder level i, the input feature map Fi is decomposed into its low- and high-frequency components by the Spectral Decomposition Block (SDB) shown in Figure 3 (a) <ref:2609.02839#pg2> This decomposition is implemented via a smooth Fourier mask based on a given cutoff parameter M(r) <ref:2609.02839#pg2> The higher the cutoff ratio, the larger the circle we use to eliminate frequencies from the center, thus, higher frequencies will be kept <ref:2609.02839#pg2> To obtain the low-frequencies, we apply the inverse mask <ref:2609.02839#pg2> The refined component is Fˆl i <ref:2609.02839#pg2> The encoder output at each scale is obtained by summing both components, F enc i = Fˆl i + F h i <ref:2609.02839#pg2> This design explicitly preserves high-frequency information throughout the encoding process, preventing the loss of fine structural details that are critical for weather artifact removal <ref:2609.02839#pg2> Spectral Fusion and Refinement A key contribution of FReSH is the Spectral Harmonization Block (SHB), which replaces standard additive skip connections with a frequency-aware fusion mechanism <ref:2609.02839#pg2> At each decoder level, the upsampled feature map X is decomposed in the Fourier domain using the same mask used in SDB <ref:2609.02839#pg2> The computed frequencies are summed with the corresponding residual frequencies from the same-level encoder <ref:2609.02839#pg2> This frequency-aware fusion enables the decoder to selectively integrate structural LF information and HF detail from the encoder at each scale, rather than naively merging all feature channels <ref:2609.02839#pg2> Performance and Efficiency Gains FReSH-IR achieves restoration quality competitive with leading AIO methods while requiring 80% fewer parameters, up to 90% fewer MACs, and running with 5× faster inference than top-performing approaches such as SSGFormer <ref:2609.02839#pg2> In Table 1, FReSH-IR uses 52× and 10× fewer parameters than the fastest solutions MWFormer
58: and TransWeather
41: , respectively <ref:2609.02839#pg2> This results in lower memory requirements, while our overall performance is superior, +1.67dB and +0.08dB <ref:2609.02839#pg2> FReSH-IR offers an optimal trade-off in terms of memory, speed, and performance <ref:2609.02839#pg2> Downstream Task Benefits In Table 3 we show the improvement of these tasks performance across the Rainy Cityscapes dataset, where applying our restoration method as a preprocessing step yields consistent improvements over the degraded input <ref:2609.02839#pg2> We also provide qualitative results on detection in Figure 6 <ref:2609.02839#pg2> These experiments confirm that FReSH-IR not only restores visual quality but also yields measurable benefits for downstream perception tasks, particularly in safety-critical scenarios such as autonomous driving <ref:2609.02839#pg2> Ablation Study Insights The ablation study on the components of FReSH-IR showed that processing low frequencies in the encoder is more effective <ref:2609.02839#pg2> Furthermore, we found that restoring high frequencies at the upper levels of the decoder is more effective than processing HF with downsampled features <ref:2609.02839#pg2> The results also indicate that the Dual-Attention Module's significance is underscored when replaced by a standard NAFBlock
2: , which resulted in decreased performance <ref:2609.02839#pg2> We believe this work provides a practical pathway towards real-world, on-device image restoration, and encourages further research into efficiency-first frameworks that treat deployment constraints as a core design objective <ref:2609.02839#pg2> The frequency-aware architecture can recover most of the quality of state-of-the-art methods at a fraction of the resources <ref:2609.02839#pg2> This trade-off is not a limitation but a deliberate design decision <ref:2609.02839#pg2> The paper concludes that FReSH-IR is an efficient all-in-one weather restoration method designed for edge applications <ref:2609.02839#pg2> Limitations and Future Work The performance of the method on real-world scenes is still limited for demanding downstream tasks, such as autonomous driving <ref:2609.02839#pg2> This is common among all the methods and can be attributed to the limited availability of realistic training datasets <ref:2609.02839#pg2> Future work should focus on the creation of these datasets rather than on architectural development <ref:2609.02839#pg2> The paper also includes results showing that only FReSH-IR and WGWS
59: handle resolutions of at least 1440p <ref:2609.02839#pg2> FReSH-IR demonstrates superior scalability in inference times relative to image resolution compared to other methods <ref:2609.02839#pg2> The alternative methods encounter Out-of-Memory (OOM) errors <ref:2609.02839#pg2> Experimental Setup Details For a fair comparison, we use the standard benchmark for all-in-one multi-weather restoration, following
30: <ref:2609.02839#pg2> It is a combination of three datasets: RainDrop
36: , OutdoorRain
29: , and Snow100K
33: <ref:2609.02839#pg2> We trained the model for 500 epochs using 384 × 384 crops and a batch size of 16 <ref:2609.02839#pg2> The initial learning rate was set to 1e−3 and we used the AdamW optimizer with β1, β2 = 0.9 and a weight decay of 1e−3 <ref:2609.02839#pg2> Dataset Composition Training dataset.
Tom: Following previous studies
30, 41: , we use the AllWeather set <ref:2609.02839#pg2> It contains a subset of 9, 000 images from Outdoor-Rain
29: , which encompasses synthetic images with haze and rain-streaks altogether; 1, 069 images from RaindropData
36: , containing real raindrop images; and 9, 000 images from the Snow100K
33: training dataset, featuring synthetic images affected by snow <ref:2609.02839#pg2> Rain + Fog test set.
Jane: For both deraining and dehazing, we use the Test1 subset from Outdoor-Rain
29: <ref:2609.02839#pg2> Raindrop test set.
Tom: The dataset used for testing de-raindrop is the Raindrop A subset from RaindropData
36: <ref:2609.02839#pg2> Snow test set.
Title and authors: Tom: So we're looking at this paper today called "Efficient All-in-One Weather Restoration using Spectral Harmonization" by Garrido-Mellado, Feijoo, Cui, Garcia, and Conde. Jane, what should we say about the title itself?
Jane: Well it sounds a bit heavy with all those technical terms like spectral harmonization and frequency reconstruction. But at its heart it’s trying to be an efficient way to fix images when they have weather stuff on them. It suggests they are breaking down the image into different parts based on how fast the details change across the picture.
Lu: Exactly, Jane, so instead of treating a whole image uniformly, this method explicitly splits those feature representations into high and low frequency components at every scale in their network architecture <ref:2609.02839#pg2>. That decomposition is key to their approach.
Meng: I'm more interested in the authors, because they claim they got comparable restoration quality to these much bigger transformer models, but with way fewer parameters and operations <ref:2609.02839#pg2>. If that's true, that’s a big deal for deployment right now.
Tom: That's what I was thinking too, Meng. The claim is they achieved similar quality to transformer-based models but used eighty percent fewer parameters and operations <ref:2609.02839#pg2>. That points toward something really practical for edge devices.
Jane: So, what this paper is promising is that you don't have to sacrifice quality just because you want a faster model, especially when dealing with complex weather effects like rain or snow <ref:2609.02839#pg1>. It’s about finding that balance.
Lu: And the authors are using an asymmetric UNet architecture, which is a standard structure but they're modifying it to handle these frequency components separately before recombining them <ref:2609.02839#pg2>. They are essentially trying to give the network specialized tools for handling different types of image corruption.
Meng: Specialized tools are good, but what about the cost? I need to know if this complexity actually translates into a smaller footprint when we put it into a production pipeline <ref:2609.02839#pg2>.
The paper's summary: Tom: Okay, so to summarize what they are doing with "Efficient All-in-One Weather Restoration using Spectral Harmonization," they are proposing a novel lightweight all-in-one restoration method <ref:2609.02839#pg1>. They achieve this by decomposing the features into high and low frequency components at each scale of a hierarchical encoder-decoder structure <ref:2609.02839#pg1>.
Jane: That decomposition is done by combining spectral decomposition with spatial processing through Fourier based skip connections, which lets them capture complementary frequency information without losing the actual spatial detail of the image <ref:2609.02839#pg5>. It’s like looking at a picture both in broad strokes and zooming in on the fine details simultaneously.
Lu: They do this by using a smooth Fourier mask based on a cutoff parameter M of r to separate these components, and then they refine the low frequency component through residual blocks before merging everything back together <ref:2609.02839#pg2>. That's how they build up the restoration.
Meng: So, what does this actually mean for me on a deployment side? Does this mean the inference time is drastically cut down when we run it on something like a Jetson Orin NX <ref:2609.02839#pg2>? Because I'm worried about latency when things get busy.
Tom: The paper claims they achieve similar restoration quality with eighty percent fewer parameters and operations compared to transformer-based models <ref:2609.02839#pg2>. That suggests a significant win for speed and memory usage <ref:2609.02839#pg1>.
Jane: It seems they are explicitly designing the network to be lightweight while making sure it still picks up the fine structural details that are crucial for removing those tricky weather artifacts <ref:2609.02839#pg2>. That’s the whole point of keeping both frequencies intact.
Lu: Right, and they use a Dual-Attention Module at the bottleneck to capture long-range spatial dependencies, which helps guide the process before it goes into the decoder <ref:2609.02839#pg3>. That’s a different mechanism than what some other papers might use for that part of the network.
Meng: Long-range dependencies are cool, but if that module adds too much overhead during inference, we're back to square one <ref:2609.02839#pg2>. I need concrete numbers on how much that attention mechanism slows things down.
The paper's improvements: Tom: One of the big improvements they highlight is their Spectral Harmonization Block, or SHB, which replaces standard additive skip connections with a frequency-aware fusion mechanism <ref:2609.02839#pg2>.
Jane: That’s interesting because it means instead of just naively merging all the features from the encoder and decoder at each step, the decoder can selectively integrate structural low frequency information and high frequency detail based on what it needs for that specific layer <ref:2609.02839#pg2>.
Lu: That selectivity is really powerful. They decompose the upsampled feature map in the Fourier domain using the same mask they used in the Spectral Decomposition Block to sum it with residual frequencies from the encoder at each level <ref:2609.02839#pg3>. It’s not just adding things up; it’s a frequency-aware fusion <ref:2609.02839#pg1>.
Meng: So they aren't blindly merging everything; they are making a calculated choice about which frequency information to keep or discard at each step, which is smart for efficiency <ref:two thousand six hundred nine point zero two eight three nine#pg1. That’s the kind of design I can actually see in production systems.
Tom: And there's also this idea of using different masks in each level of the Spectral Decomposition Block to improve restoration performance <ref:2609.02839#pg5>. It shows they are tuning the frequency separation dynamically, which should help them handle different weather artifacts better.
Jane: That sounds like a smart way to address the problem that some other methods might treat all frequencies the same way, which limits their ability to fix localized stuff like raindrops or snowflakes <ref:2609.02839#pg2>.
Lu: And they also put some specific attention on where they focus their restoration efforts, showing that processing low frequencies in the encoder is more effective for performance <ref:2609.02839#pg3>. That tells us where the network should spend its effort to get the most bang for our buck.
Meng: Okay, so if we’re focusing on low frequencies in the encoder and using this smart fusion block, it sounds like they are optimizing the computational budget effectively <ref:two thousand six hundred nine point zero two eight three nine#pg1. That's what I look for when I'm trying to ship something fast.
Conclusion: Tom: So, to wrap up on "Efficient All-in-One Weather Restoration using Spectral Harmonization," this method is an all-in-one restoration method designed specifically for edge applications <ref:2609.02839#pg1>. They use the spectral decomposition and those frequency branches to decompose and fuse spectral information at every scale, achieving restoration quality competitive with much heavier models at up to ninety percent lower computational cost in terms of memory, operations, and runtime <ref:2609.02839#pg1>.
Jane: It really boils down to this idea that they aren't making a massive model just because they can; they are using this frequency-aware architecture to recover most of the quality of state-of-the-art methods at a fraction of the resources <ref:2609.02839#pg1>. It’s a deliberate design decision to trade off resources for speed and memory, which we see more and more in real-world systems.
Lu: From my view, the main implication is that this approach provides a practical pathway toward real-world on-device image restoration by treating deployment constraints as a core design objective <ref:2609.02839#pg1>. It shows that you can get results close to the state-of-the-art with far less than half the resources <ref:two thousand six hundred nine point zero two eight three nine#pg1.
Meng: From a production standpoint, I'm just happy to hear that they are tackling the constraints of deployment head-on, but I still need to know how this performs on really messy, real traffic data that we haven't seen before <ref:two thousand six hundred nine point zero two eight three nine#pg1.
Lalam: Based on what I’ve processed from the paper, I am very sure that FReSH-IR is an efficient all-in-one weather restoration method designed for edge applications <ref:2609.02839#pg1>. It’s a solid solution for when you need something fast and resource efficient on the go.
Tom: So, we're done with this one. We’ve looked at how this Spectral Harmonization method works, how it cuts the computational cost by eighty percent compared to transformer-based models, and how it compares to other methods in terms of speed <ref:2609.02839#pg1>.
Jane: It’s a neat solution for someone who is trying to get high quality results without needing massive computational budgets <ref:two thousand six hundred nine point zero two eight three nine#pg1. We’ve seen how this Spectral Harmonization method works, how it cuts the computational cost by eighty percent compared to transformer-based models, and how it compares to other methods in terms of speed.
Lu: Yeah, we've established that they are explicitly decomposing features into high and low frequency components at each scale in their hierarchical encoder-decoder architecture for this paper <ref:two thousand six hundred nine point zero two eight three nine#pg1.
Meng: Alright, the next time we talk about a paper like this, I want to hear more on how it actually handles those real-world edge cases without running into OOM errors <ref:2609.02839#pg2>.
Cidaut AI, Fundacion Cidaut, Spain · Technical University of Munich, Germany
cs.CV
Submitted: 2026-09-02
Updated: 2026-11-16
Comments: WACV 2027
License: http://creativecommons.org/licenses/by-nc-nd/4.0/
Importance score: 100/100
The gist: Efficient All-in-One Weather Restoration using Spectral Harmonization presents FReSH-IR, a novel lightweight all-in-one restoration method that explicitly decomposes feature representations into…
Key concepts
- Spectral Decomposition Block (SDB)
- This block splits the input image features into low-frequency and high-frequency parts using a smooth Fourier mask. This allows the model to isolate weather artifacts, which are often concentrated in high frequencies, from the underlying structural details of the clean image.
- Spectral Harmonization Block (SHB)
- This block replaces standard connections by fusing features based on their frequency content. It enables the decoder to selectively integrate structural low-frequency information and fine high-frequency details from different encoder levels, ensuring a more precise reconstruction.
- Dual-Attention Module (DAM)
- This module captures long-range spatial dependencies in the bottleneck using attention mechanisms based on average and max pooling. It helps the network identify important spatial locations within the features to modulate subsequent processing paths for enhanced detail recovery.
- Frequency Transformer Blocks (FTBs)
- These blocks use different processing strategies for each frequency component: LF branches use parallel dilated convolutions for multi-scale structural information, while HF branches employ transformer blocks to capture long-range dependencies in fine details.
Terminology
Summary
Efficient All-in-One Weather Restoration using Spectral Harmonization presents FReSH-IR, a novel lightweight all-in-one restoration method that explicitly decomposes feature representations into high- and low-frequency components at each scale of a hierarchical encoder-decoder architecture <ref:2609. p. 0 In this paper, we present
Frequency Reconstruction via Spectral Harmonization, a
novel lightweight all-in-one restoration method that explicitly decomposes feature representations into high- and
low-frequency components at each scale of a hierarchical encoder-decoder architecture. By combining spectral decomposition with spatial processing through Fourierbased skip connections, FReSH-IR captures complementary frequency information without sacrificing spatial detail. In this paper, we present
Frequency Reconstruction via Spectral Harmonization, a
novel lightweight all-in-one restoration method that explicitly decomposes feature representations into high- and
low-frequency components at each scale of a hierarchical encoder-decoder architecture. By combining spectral decomposition with spatial processing through Fourierbased skip connections, FReSH-IR captures complementary frequency information without sacrificing spatial detail. By combining spectral decomposition with spatial processing through Fourierbased skip connections, FReSH-IR captures complementary frequency information without sacrificing spatial detail. Our approach achieves similar restoration quality with
80% fewer parameters and operations than transformerbased models. Our approach achieves similar restoration quality with
80% fewer parameters and operations than transformerbased models. Extensive experiments demonstrate that
our method offers a great efficiency-performance tradeoff, highlighting its practical applications in constrainedresource systems.Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Methodology of FReSH-IR
The proposed FReSH-IR follows a U-shaped CNN-based backbone <ref:2609.02839#pg2> Given a corrupted image I ∈ R 3×H×W, a 3×3 convolutional layer first projects the input into shallow feature embeddings of size C × H × W <ref:2609. p. 2 Given a corrupted image I ∈ R
3×H×W , where 3 denotes the number of channels, and H and W denote height and width, respectively. A 3×3 convolutional layer first projects the input into shallow feature embeddings of size C × H × W. A 3×3 convolutional layer first projects the input into shallow feature embeddings of size C × H × W. These embeddings are processed through a hierarchical
encoder that progressively downsamples the features while
explicitly decomposing them into low- and high-frequency
components at each scale. These embeddings are processed through a hierarchical
encoder that progressively downsamples the features while
explicitly decomposing them into low- and high-frequency
components at each scale. The low-frequency component is
refined through a stack of Residual Blocks, and both components are recombined before downsampling. The low-frequency component is
refined through a stack of Residual Blocks, and both components are recombined before downsampling. At the bottleneck, a Dual-Attention Module (DAM)
captures long-range spatial dependencies, which, as shown
in [9], is particularly beneficial for homogeneous degradations such as haze, which affects broad spatial regions. At the bottleneck, a Dual-Attention Module (DAM)
captures long-range spatial dependencies, which, as shown
in [9], is particularly beneficial for homogeneous degradations such as haze, which affects broad spatial regions. A carefully designed decoder progressively recovers the
features, recombining the upsampled features with the corresponding encoder features. A carefully designed decoder progressively recovers the
features, recombining the upsampled features with the corresponding encoder features. Each block processes the high
and low components separately, only fusing them during the
last block of each level.Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Frequency-Aware Feature Extraction
At each encoder level i, the input feature map Fi is decomposed into its low- and high-frequency components by the Spectral Decomposition Block (SDB) shown in Figure 3 (a) <ref:2609.02839#pg2> This decomposition is implemented via a smooth Fourier mask based on a given cutoff parameter M(r) <ref:2609. p. 2 Each block processes the high
and low components separately, only fusing them during the
last block of each level. Frequency-Aware Encoder
At each encoder level i, the input feature map Fi
is decomposed into its low- and high-frequency components by the
Spectral Decomposition Block (SDB) shown in Figure 3
(a), implemented via a smooth Fourier mask based on a
given cutoff parameter, following:
M(r) =
0 r ≤ r0 − 2. 15r0 and α is the cutoff. Since the lower frequencies concentrate in the middle and the higher frequencies correspond to the spectrum
edges, the higher the cutoff ratio, the larger the circle we use
to eliminate frequencies from the center, thus, higher frequencies will be kept. Since the lower frequencies concentrate in the middle and the higher frequencies correspond to the spectrum
edges, the higher the cutoff ratio, the larger the circle we use
to eliminate frequencies from the center, thus, higher frequencies will be kept. To obtain the low-frequencies, we apply the inverse mask. The low-frequency component is then refined through a
stack of Residual Blocks introduced in [2], which perform
lightweight attention-based feature enhancement. The refined component is Fˆl
i
. The refined component is Fˆl
i
. The encoder output at each scale is obtained by summing both components, F
enc
i = Fˆl
i + F
h
i
. The encoder output at each scale is obtained by summing both components, F
enc
i = Fˆl
i + F
h
i
. This design
explicitly preserves high-frequency information throughout
the encoding process, preventing the loss of fine structural
details that are critical for weather artifact removal.Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Spectral Fusion and Refinement
A key contribution of FReSH is the Spectral Harmonization Block (SHB), which replaces standard additive skip connections with a frequency-aware fusion mechanism <ref:2609.02839#pg2> At each decoder level, the upsampled feature map X is decomposed in the Fourier domain using the same mask used in SDB <ref:2609. p. 2 Spectral Skip Connections
A key contribution of FReSH is the Spectral Harmonization Block (SHB), which replaces standard additive skip
connections with a frequency-aware fusion mechanism. At
each decoder level, the upsampled feature map X is decomposed in the Fourier domain using the same mask used
3 The input is divided into frequency components similar
to the SDB. The computed frequencies are summed with the corresponding residual frequencies from the same-level encoder. The decomposed components
are then fused with their corresponding encoder skip features. This frequency-aware fusion enables the decoder to
selectively integrate structural LF information and HF detail
from the encoder at each scale, rather than naively merging
all feature channels.Where the paper says it
Where the paper says it
Where the paper says it
Performance and Efficiency Gains
FReSH-IR achieves restoration quality competitive with leading AIO methods while requiring 80% fewer parameters, up to 90% fewer MACs, and running with 5× faster inference than top-performing approaches such as SSGFormer <ref:2609.02839#pg2> In Table 1, FReSH-IR uses 52× and 10× fewer parameters than the fastest solutions MWFormer [58] and TransWeather [41], respectively <ref:2609. p. 4 Quantitative Results
In Table 1, we provide a comparative analysis of distortion
metrics applied to both synthetic and real datasets. Our
FReSH-IR uses 52× and 10× fewer parameters than the
fastest solutions MWFormer [58] and TransWeather [41],
respectively. Our
FReSH-IR uses 52× and 10× fewer parameters than the
fastest solutions MWFormer [58] and TransWeather [41],
respectively. This results in lower memory requirements,
while our overall performance is superior, +1. This results in lower memory requirements,
while our overall performance is superior, +1. 67dB and
+0. To address the challenges posed by adverse weather
conditions, several studies have explored image restoration techniques in the frequency domain. AIRFormer [14],
a general-purpose AIO model, constructs a frequencyguided transformer encoder by incorporating wavelet-based
prior information to enhance feature extraction, leveraging
1
arXiv:2609. We highlight the methods capable to process
images under 30ms . FReSH-IR offers an optimal trade-off in terms of memory, speed, and performance.Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Downstream Task Benefits
In Table 3 we show the improvement of these tasks performance across the Rainy Cityscapes dataset, where applying our restoration method as a preprocessing step yields consistent improvements over the degraded input <ref:2609.02839#pg2> We also provide qualitative results on detection in Figure 6 <ref:2609. p. 5 In Table 3 we show the improvement of these tasks performance across the Rainy Cityscapes dataset, where applying
our restoration method as a preprocessing step yields consistent improvements over the degraded input. We also provide qualitative results on detection in Figure 6. By comparing the detector’s predictions on degraded
images against those obtained after restoring them with our
method, we observe clear improvements in both detection
confidence and the number of correctly identified objects. These experiments confirm that FReSH-IR not only restores visual quality but also yields measurable benefits for
downstream perception tasks, particularly in safety-critical
scenarios such as autonomous driving.Where the paper says it
Where the paper says it
Ablation Study Insights
The ablation study on the components of FReSH-IR showed that processing low frequencies in the encoder is more effective <ref:2609.02839#pg2> Furthermore, we found that restoring high frequencies at the upper levels of the decoder is more effective than processing HF with downsampled features <ref:2609. p. 7 This finding indicates that processing HF with
downsampled features is not an effective method for image restoration. Instead, our network design prioritizes the
restoration of HF at the upper levels of the decoder, as illustrated in the architecture in Figure 2.Where the paper says it
Conclusion on Design Philosophy
We believe this work provides a practical pathway towards real-world, on-device image restoration, and encourages further research into efficiency-first frameworks that treat deployment constraints as a core design objective <ref:2609.02839#pg2> The frequency-aware architecture can recover most of the quality of state-of-the-art methods at a fraction of the resources <ref:2609. p. 7 Through
the Spectral Harmonization Block and specialized frequency branches, FReSH-IR decomposes and fuses spectral information at each scale, achieving restoration quality competitive with far heavier models at up to 90% lower
computational cost in terms of memory, operations, and
runtime. This trade-off is not a limitation but a deliberate
design decision: frequency-aware architecture can recover
most of the quality of state-of-the-art methods at a fraction
of the resources. Through
the Spectral Harmonization Block and specialized frequency branches, FReSH-IR decomposes and fuses spectral information at each scale, achieving restoration quality competitive with far heavier models at up to 90% lower
computational cost in terms of memory, operations, and
runtime. This trade-off is not a limitation but a deliberate
design decision: frequency-aware architecture can recover
most of the quality of state-of-the-art methods at a fraction
of the resources. Through
the Spectral Harmonization Block and specialized frequency branches, FReSH-IR decomposes and fuses spectral information at each scale, achieving restoration quality competitive with far heavier models at up to 90% lower
computational cost in terms of memory, operations, and
runtime. This trade-off is not a limitation but a deliberate
design decision: frequency-aware architecture can recover
most of the quality of state-of-the-art methods at a fraction
of the resources. Future work should
focus on the creation of these datasets rather than on architectural development. Conclusion
We presented FReSH-IR, an efficient all-in-one weather
restoration method designed for edge applications. Future work should
focus on the creation of these datasets rather than on architectural development. Conclusion
We presented FReSH-IR, an efficient all-in-one weather
restoration method designed for edge applications.Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Limitations and Future Work
The performance of the method on real-world scenes is still limited for demanding downstream tasks, such as autonomous driving <ref:2609.02839#pg2> This is common among all the methods and can be attributed to the limited availability of realistic training datasets <ref:2609. p. 7 Limitations
The performance of the method on real-world scenes is
still limited for demanding downstream tasks, such as autonomous driving. This is common among all the methods
and can be attributed to the limited availability of realistic
training datasets. Compared to other tasks, such as lowlight image enhancement [15, 48], collecting real samples
for weather restoration is quite difficult, which remains one
of the main challenges in this field. Future work should
focus on the creation of these datasets rather than on architectural development. Existing restoration methods require large computational budgets, struggling to process high-resolution images and
handle different degradations. In this paper, we present
Frequency Reconstruction via Spectral Harmonization, a
novel lightweight all-in-one restoration method that explicitly decomposes feature representations into high- and
low-frequency components at each scale of a hierarchical encoder-decoder architecture. A study on inference times at different
resolutions is included in the Supplementary Material. Ablation Studies
To demonstrate that the FReSH-IR architecture is constructed using components that exhibit optimal performance, we conducted a series of ablation studies on various components, as presented in Table 4. We highlight the methods capable to process
images under 30ms . FReSH-IR offers an optimal trade-off in terms of memory, speed, and performance. We highlight the methods capable to process
images under 30ms . FReSH-IR offers an optimal trade-off in terms of memory, speed, and performance.Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Experimental Setup Details
For a fair comparison, we use the standard benchmark for all-in-one multi-weather restoration, following [30] <ref:2609.02839#pg2> It is a combination of three datasets: RainDrop [36], OutdoorRain [29], and Snow100K [33] <ref:2609. p. 4 Implementation Details
For a fair comparison, we use the standard benchmark for
all-in-one multi-weather restoration, following [30]. It is
a combination of three datasets: RainDrop [36], OutdoorRain [29], and Snow100K [33]. Similarly, we used the RainDrop test dataset [36], Test1
dataset from Outdoor-Rain [29], and Snow100K-L testset
[33] for testing. We trained the model for 500 epochs using
384 × 384 crops and a batch size of 16. We trained the model for 500 epochs using
384 × 384 crops and a batch size of 16. The initial learning rate was set to 1e
−3
and we used the AdamW optimizer
with β1, β2 = 0. The initial learning rate was set to 1e
−3
and we used the AdamW optimizer
with β1, β2 = 0. 9 and a weight decay of 1e
−3
.Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Dataset Composition
Training dataset. Following previous studies [30, 41], we use the AllWeather set <ref:2609. p. 0 To address the challenges posed by adverse weather
conditions, several studies have explored image restoration techniques in the frequency domain. AIRFormer [14],
a general-purpose AIO model, constructs a frequencyguided transformer encoder by incorporating wavelet-based
prior information to enhance feature extraction, leveraging
1
arXiv:2609. The training set encompasses 1, 069 images from RainDrop, 9, 000 images
from Outdoor-Rain, and 9, 000 from Snow100K. RainDrop
comprises real raindrop images. RainDrop
comprises real raindrop images. Outdoor-Rain contains
synthetic images degraded by both fog and rain streaks. Snow100K features synthetic images afflicted by snow. Similarly, we used the RainDrop test dataset [36], Test1
dataset from Outdoor-Rain [29], and Snow100K-L testset
[33] for testing. Snow100K features synthetic images afflicted by snow. Similarly, we used the RainDrop test dataset [36], Test1
dataset from Outdoor-Rain [29], and Snow100K-L testset
[33] for testing. Snow100K features synthetic images afflicted by snow. Similarly, we used the RainDrop test dataset [36], Test1
dataset from Outdoor-Rain [29], and Snow100K-L testset
[33] for testing. Snow100K features synthetic images afflicted by snow. Similarly, we used the RainDrop test dataset [36], Test1
dataset from Outdoor-Rain [29], and Snow100K-L testset
[33] for testing. Snow100K features synthetic images afflicted by snow. Similarly, we used the RainDrop test dataset [36], Test1
dataset from Outdoor-Rain [29], and Snow100K-L testset
[33] for testing.Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Frequency Analysis Motivation
This analysis is motivated by the observation that predominant weather artifacts (e.g., snowflakes, raindrops, etc) mainly appear in the high-frequency domain of degraded images <ref:2609. p. 1 These methods demonstrate that operating in the spectral domain can effectively capture weather-related degradation patterns. However, existing frequency-domain
restoration approaches either process the spectrum globally
or apply identical operations to all components [10, 20, 57],
limiting their ability to handle spatially localized artifacts
like raindrops or snowflakes. In early layers, where low-level degradation patterns are most salient, we employ FFT-based decomposition
with stricter masks (i. , high cutoff values) to achieve precise isolation of weather artifacts. These methods demonstrate that operating in the spectral domain can effectively capture weather-related degradation patterns. However, existing frequency-domain
restoration approaches either process the spectrum globally
or apply identical operations to all components [10, 20, 57],
limiting their ability to handle spatially localized artifacts
like raindrops or snowflakes. traction, which, despite operating on deep feature representations, retains the ability to capture structural image edges. This property is particularly advantageous for the reconstruction of fine-grained image details in the later stages of
the restoration process, where preserving structural information is critical for achieving faithful outputs.Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Additional Efficiency Studies
To remark the efficiency potential of our proposal, we include the image runtime study conducted on a 24GB RTX 4090 GPU <ref:2609.02839#pg2> We conduct the same study on a 16GB Jetson Orin NX <ref:2609. p. 0 To address the challenges posed by adverse weather
conditions, several studies have explored image restoration techniques in the frequency domain. AIRFormer [14],
a general-purpose AIO model, constructs a frequencyguided transformer encoder by incorporating wavelet-based
prior information to enhance feature extraction, leveraging
1
arXiv:2609. A study on inference times at different
resolutions is included in the Supplementary Material. Ablation Studies
To demonstrate that the FReSH-IR architecture is constructed using components that exhibit optimal performance, we conducted a series of ablation studies on various components, as presented in Table 4. A study on inference times at different
resolutions is included in the Supplementary Material. Ablation Studies
To demonstrate that the FReSH-IR architecture is constructed using components that exhibit optimal performance, we conducted a series of ablation studies on various components, as presented in Table 4.Where the paper says it
Where the paper says it
Where the paper says it
Ablation Study Summary
Table 7 (a) shows that processing only the high frequencies on the encoder resulted in a significant decrease in performance <ref:2609.02839#pg2> The results of these studies are presented in Table 7 (a) <ref:2609. p. 5 A study on inference times at different
resolutions is included in the Supplementary Material. Ablation Studies
To demonstrate that the FReSH-IR architecture is constructed using components that exhibit optimal performance, we conducted a series of ablation studies on various components, as presented in Table 4. A study on inference times at different
resolutions is included in the Supplementary Material. Ablation Studies
To demonstrate that the FReSH-IR architecture is constructed using components that exhibit optimal performance, we conducted a series of ablation studies on various components, as presented in Table 4. A study on inference times at different
resolutions is included in the Supplementary Material. Ablation Studies
To demonstrate that the FReSH-IR architecture is constructed using components that exhibit optimal performance, we conducted a series of ablation studies on various components, as presented in Table 4. A study on inference times at different
resolutions is included in the Supplementary Material. Ablation Studies
To demonstrate that the FReSH-IR architecture is constructed using components that exhibit optimal performance, we conducted a series of ablation studies on various components, as presented in Table 4. A study on inference times at different
resolutions is included in the Supplementary Material. Ablation Studies
To demonstrate that the FReSH-IR architecture is constructed using components that exhibit optimal performance, we conducted a series of ablation studies on various components, as presented in Table 4. A study on inference times at different
resolutions is included in the Supplementary Material. Ablation Studies
To demonstrate that the FReSH-IR architecture is constructed using components that exhibit optimal performance, we conducted a series of ablation studies on various components, as presented in Table 4. During the machine learning era,
CNN and GAN-based approaches later improved performance by learning data-driven degradation priors [11, 36,
47]. The authors of [37] showed significant advancements
in this field by introducing edge information. A carefully designed decoder progressively recovers the
features, recombining the upsampled features with the corresponding encoder features. Each block processes the high
and low components separately, only fusing them during the
last block of each level. does not yield any improvement, thereby reinforcing our
strategy of initially restoring LF in the encoder and HF in
the decoder. To demonstrate the efficacy of the dual attention module, we replaced this component with a standard
NAFBlock [2], which resulted in decreased performance. Overall Framework
The proposed FReSH-IR follows a U-shaped CNN-based
backbone, as illustrated in Figure 2. Given a corrupted image I ∈ R
3×H×W , where 3 denotes the number of channels, and H and W denote height and width, respectively.Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Qualitative Visual Comparison
In Figure 4 we show visual results on both synthetic and real-world benchmarks compared to several all-in-one weather restoration methods such as TransWeather [41], WGWS [59] and Histoformer [40] <ref:2609.02839#pg2> Our visual results are comparable to SOTA methods on synthetic (Fig. p. 4 Qualitative Results
In this section, we show visual results on both synthetic
and real-world benchmarks compared to several all-in-one
weather restoration methods such as TransWeather [41],
WGWS [59] and Histoformer [40]. Our visual results are
comparable to SOTA methods on synthetic (Fig. Our visual results are
comparable to SOTA methods on synthetic (Fig. 4) and realworld benchmarks (Fig. 4) and realworld benchmarks (Fig. 5) while saving 80% of computational cost against SSGFormer [18]. Additionally, in Table 2 we include a set of experiments in a
real-world benchmark, demonstrating that we achieve competitive results also in real-world scenarios. Qualitative Results
In this section, we show visual results on both synthetic
and real-world benchmarks compared to several all-in-one
weather restoration methods such as TransWeather [41],
WGWS [59] and Histoformer [40]. Through
the Spectral Harmonization Block and specialized frequency branches, FReSH-IR decomposes and fuses spectral information at each scale, achieving restoration quality competitive with far heavier models at up to 90% lower
computational cost in terms of memory, operations, and
runtime. This trade-off is not a limitation but a deliberate
design decision: frequency-aware architecture can recover
most of the quality of state-of-the-art methods at a fraction
of the resources. Additionally, in Table 2 we include a set of experiments in a
real-world benchmark, demonstrating that we achieve competitive results also in real-world scenarios. Qualitative Results
In this section, we show visual results on both synthetic
and real-world benchmarks compared to several all-in-one
weather restoration methods such as TransWeather [41],
WGWS [59] and Histoformer [40].Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Conclusion
We presented FReSH-IR, an efficient all-in-one weather restoration method designed for edge applications <ref:2609.02839#pg2> Through the Spectral Harmonization Block and specialized frequency branches, FReSH-IR decomposes and fuses spectral information at each scale, achieving restoration quality competitive with far heavier models at up to 90% lower computational cost in terms of memory, operations, and runtime <ref:2609. p. 7 Conclusion
We presented FReSH-IR, an efficient all-in-one weather
restoration method designed for edge applications. Through
the Spectral Harmonization Block and specialized frequency branches, FReSH-IR decomposes and fuses spectral information at each scale, achieving restoration quality competitive with far heavier models at up to 90% lower
computational cost in terms of memory, operations, and
runtime. pathway towards real-world, on-device image restoration,
and encourages further research into efficiency-first frameworks that treat deployment constraints as a core design objective. References
[1] Bolun Cai, Xiangmin Xu, Kui Jia, Chunmei Qing, and
Dacheng Tao. pathway towards real-world, on-device image restoration,
and encourages further research into efficiency-first frameworks that treat deployment constraints as a core design objective. References
[1] Bolun Cai, Xiangmin Xu, Kui Jia, Chunmei Qing, and
Dacheng Tao.Where the paper says it
Where the paper says it
Where the paper says it
Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.
Improvements for AI systems
-
Spectral Harmonization Block (SHB) allows for
frequency-aware fusion enables the decoder to selectively integrate structural LF information and HF detail from the encoder at each scale, rather than naively merging all feature channels.
This enables more precise reconstruction by choosing which frequency component to emphasize based on local image content. -
The use of three complementary losses Ltotal = λpxLpx + λenhLenh + λf req ensures supervision at the pixel, perceptual, and spectral levels simultaneously. This joint supervision prevents the network from suppressing fine details during degradation removal through the Fourier edge loss term Lf req = Fhfenc − Ihfgt 2/2.
-
The Dual-Attention Module (DAM) processes features sequentially by applying spatial attention based on AvgPool and MaxPool, followed by channel refinement via depth-wise convolutions. This mechanism helps
identify key spatial locations
which are then used to modulate the first path of the input feature map Fs, leading to a spatially enhanced feature map Fs. -
The Frequency Transformer Blocks (FTBs) allow the LF branch to process features through parallel dilated depthwise convolutions for multi-scale structural information, while the HF branch uses a simple transformer-based block for long-range dependencies within fine-grained features. This separation ensures that both components are
fully exploited before being merged back into a unified representation.
-
The regularization strategy involves setting different masks in each level of the Spectral Decomposition Block (SDB) to improve restoration performance. This practice shows the
use of different masks in each level, improving the restoration performance.
Abstract
Adverse weather conditions such as rain, haze, and snow significantly degrade image quality, posing challenges for both human perception and physical AI. Existing restoration methods require large computational budgets, struggling to process high-resolution images and handle different degradations. In this paper, we present Frequency Reconstruction via Spectral Harmonization, a novel lightweight all-in-one restoration method that explicitly decomposes feature representations into high- and low-frequency components at each scale of a hierarchical encoder-decoder architecture. By combining spectral decomposition with spatial processing through Fourier-based skip connections, FReSH-IR captures complementary frequency information without sacrificing spatial detail. Our approach achieves similar restoration quality with 80% fewer parameters and operations than transformer-based models. Extensive experiments demonstrate that our method offers a great efficiency-performance trade-off, highlighting its practical applications in constrained-resource systems.
Sources
- SnowFormer: Context Interaction Transformer with Scale-awareness for Single Image Desnowing
- WM-MoE: Weather-aware Multi-scale Mixture-of-Experts for Blind Adverse Weather Removal
Related papers
- Peak-End-Net: A Peak-End Rule Inspired Framework for Generalizable Video Aesthetic Assessment
- Sensor2Sensor: Cross-Embodiment Sensor Conversion for Autonomous Driving
- WithEveryone: Unified Planning and Identity Grounding for Group Image Generation
- SciFigure2Code: An AI-Reconstructed Benchmark for Scientific Figure-to-Code
- RepFusion: Leveraging Multimodal Priors for Denoising in Representation Space
- LucidNFT: LR-Anchored Multi-Reward Preference Optimization for Flow-Based Real-World Super-Resolution