Low-light Image Enhancement via Multi-scale Attention combined with Fourier Transform

summary

The gist

Low-light image enhancement (LLIE) aims to improve image quality and clarity in diverse and demanding low-illumination environments.

In short

This research introduces Multi-scale Attention combined with Fourier Transform (MSFT) for low-light image enhancement. The method uses a two-stage U-shaped framework combining CNNs and Transformers, guided by frequency domain information via Fourier transforms. This approach effectively restores background details and illumination in dark images, significantly outperforming existing methods.

Key concepts

Retinex decomposition
This is a technique used to separate an image into two components: the illumination (how bright the scene is) and the reflectance (the actual colors and details of objects). By separating these parts, the enhancement process can focus on correcting lighting issues independently from restoring image clarity.
Fourier Transform-Guided Multi-Scale Attention (FTG-MSA)
This core module uses frequency domain information to guide attention. It combines spatial brightness maps with low-light image amplitudes and uses FFT and cross-attention mechanisms to dynamically select and draw the most relevant features from the input data during enhancement.
Multi-Shape Synergistic Attention (MSSA)
This module is designed for multi-scale feature extraction, consisting of sequentially connected components like MSA, SCSA, and a dynamic filter network. It helps the model capture texture details across different scales while integrating information effectively into the feature space channels.

Terminology used across episodes

This episode discusses

The paper

Low-light Image Enhancement via Multi-scale Attention combined with Fourier Transform · Read on arXiv

Wenbin Dua, Jian Longa, *, Zhu Caob,c,*

Key Laboratory of Smart Manufacturing in Energy Chemical Process, Ministry of Education, East China University of Science and Technology · College of Electronics and Information Engineering, Tongji University · Shanghai Research Institute for Intelligent Autonomous Systems, Tongji University

Low-light image enhancement (LLIE) aims to improve image quality and clarity in diverse and demanding low-illumination environments. However, existing deep learning-based LLIE methods struggle to accurately capture real-world illumination and restore texture details, largely because their algorithmic strengths remain underutilized. To address these issues, we present a supervised frequency domain deep learning network for LLIE, named multi-scale attention combined with the Fourier transform (MSFT) which adopts a U-shaped, one-stage architecture that infuses guidance from low-light images into the network by channeling it through multi-scale attention. We further fuse the amplitude information from priori channels with that of the low-light image in MSFT's self-created module, and carry out multi-scale guidance along with the network. Subsequently, to better enhance the faint feature, such as fine content and textures, and to better fuse global context confidence in the decoding stage, we separately introduce a multi-shape synergistic attention and a lightweight network that effectively integrate information in high-dimensional space to embed into the superlative feature space channel containing rich texture information. Extensive experiments conducted on LOL, SID, SMID, and SDSD datasets demonstrate that MSFT significantly outperforms state-of-the-art competitors. For example, compared with Retinexformer, our method achieves a peak signal-to-noise ratio of up to 41.76 decibels on the SDSD-outdoor dataset with an increase of 11.92 decibels and a structural similarity index of 0.988 with a 13.80% improvement.

DOI: 10.1016/j.eswa.2026.132146

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: We're looking at the paper: "Low-light Image Enhancement via Multi-scale Attention combined with Fourier Transform".

Jane: Here's the gist of the paper. Low-light image enhancement (LLIE) aims to improve image quality and clarity in diverse and demanding low-illumination environments.

Tom: This paper presents a supervised frequency domain deep learning network named multi-scale attention combined with the Fourier transform (MSFT) for LLIE.

Jane: The proposed MSFT method is a two-stage U-shaped framework combining the Fourier transform with a CNN-Transformer Cross-mixed U-net framework.

Tom: This architecture involves several key components including the Retinex decomposition performed in the original image space.

Title and authors: Tom: Let's look at what they are actually trying to achieve with this "Low-light Image Enhancement via Multi-scale Attention combined with Fourier Transform" framework. It’s not just one trick; it’s a whole system designed to solve the problem of low-light image enhancement, which is often messy.

Jane: The summary points out that existing deep learning methods struggle to capture real-world illumination and restore texture details because their algorithmic strengths aren't being fully used. This paper directly addresses that by proposing a supervised frequency domain deep learning network named MSFT.

Lu: They detail the U-shaped framework, which combines a CNN-Transformer cross-mixed U-net architecture, and they use Retinex decomposition in the original image space as part of that process to get started.

Meng: The core enhancement module they focus on is called Fourier Transform-Guided Multi-Scale Attention, or FTG MSA. This module uses both the amplitude of the four channel feature brightness map and the amplitude of the low-light image in its spatial domain as guidance information.

Lalam: And to make that guidance concrete, they combine the FFT of Y with the FFT of Fn to derive a guiding factor called Yb which is reshaped after an inverse Fourier transform over Pn. That’s how they use frequency domain information directly in the network.

Tom: So, it's not just applying a filter; it’s dynamically drawing relevant features from the source data during enhancement using both FFT and cross-attention mechanisms to guide what the model focuses on.

The paper's summary: Jane: When we look at their summary, they highlight that conventional low-light techniques, like histogram equalization or Retinex methods, are too laborious and time consuming because they often neglect the processing of missing information in image black spots. This makes them fail to meet the demands for human visibility and can even compromise performance when you need good texture details.

Lu: The paper argues that their MSFT approach is proposed to highlight those texture details of the low-light background while ensuring precise illumination recovery, which they show works by using the frequency domain amplitude spectrum of a normal light image added to the low-light image before reconstructing them via inverse Fourier transform.

Meng: They show visualization results where the former successfully achieves a brightness enhancement effect similar to that under normal lighting conditions, which gives a solid, realistic basis for the idea that we can prevent overexposure in these tough scenarios.

Lalam: That means they are showing that by using the frequency domain amplitude spectrum as guidance, they can achieve a brightness enhancement effect that looks realistic without blowing out highlights or losing detail.

Tom: It seems like the main takeaway here is moving beyond simple brightness correction and actually using the frequency domain to guide the network so it learns how to recover texture information instead of just guessing a uniform light level.

The paper's improvements: Jane: They suggest several improvements in their framework, starting with Multi-Shape Synergistic Attention, or MSSA. This module is designed to accommodate multi-scale feature extraction capabilities by having three serially connected components: MSA, SCSA, and a dynamic filter network.

Lu: Specifically, the MSA module is built by parallel-connecting Dilated Square Attention and Dilated Rectangle Attention in parallel to get better scale coverage. They also mention the necessity of the FFT module because without it, performance generally drops across all datasets.

Meng: And when we look at how they optimized this for deployment, they found that removing components like CMUNeXt significantly reduced parameters down to one point one million parameters and computational volume to about seventeen point four hundred fifty-four GFLOPs while keeping strong performance on the SDSD-outdoor and SDSD-indoor datasets.

Tom: That's a huge win for deployment, because they managed to keep the accuracy high while making the model much lighter than what we usually see in these complex architectures. The ablation study confirmed that removing the FTG module caused a general decline in performance across all tested datasets, proving its necessity.

Lalam: So, to summarize their improvements, they showed that all major components are needed; specifically, removing the FFT module caused a sharp drop because it’s indispensable for capturing and reconstructing global frequency information and structural consistency of the image.

Conclusion: Jane: So wrapping up this paper on "Low-light Image Enhancement via Multi-scale Attention combined with Fourier Transform," the main implication is that we can use a frequency domain guided attention mechanism to restore degraded images while effectively enhancing background details and accurately restoring the original illumination.

Tom: It’s a reference point for constructing models that use this frequency domain guided attention mechanism to restore images, which is really interesting because it gives us a new direction for how we approach image restoration problems.

Lu: The method does aim to effectively enhance the background details of low-light images while accurately restoring the original illumination, but they also flagged a limitation: the model is limited in dealing with images polluted by strong light due to ineffective removal of light pollution noise from the exposed part.

Meng: Also, it requires a large amount of paired data for training to perform well, so if you are trying to use this for an unsupervised augmentation field without paired data right now, it’s not ideal.

Lalam: And future work they suggest is exploring Attention calculations using both the amplitude spectrum and phase spectrum to see if that leads to even superior performance in frequency domain learning.

Tom: So, that's the rundown on MSFT; it’s a powerful method for enhancing low-light images when you have good data, but we still need to tackle those noise issues and scale up the training data requirements.

More episodes

← Home