M2Retinexformer: Multi-Modal Retinexformer for Low-Light Image Enhancement
Listen
Radio episode about this paper
Transcript
Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.
Tom: We're looking at the paper: "M2Retinexformer: Multi-Modal Retinexformer for Low-Light Image Enhancement".
Jane: Here's the gist of the paper. M2Retinexformer proposes a novel framework that extends Retinexformer by incorporating depth cues, luminance priors, and semantic features within a progressive refinement pipeline.
Tom: The core innovation of the proposed method involves multi-modal fusion for low-light image enhancement.
Jane: The framework extends Retinexformer by incorporating depth, luminance, and semantic features through a Multi-Modal Cross-Attention Block (MMCAB) and an adaptive gating mechanism that balances self-attention and cross-attention based on auxiliary reliability.
Tom: This design fuses heterogeneous modality features within a modular and extensible architecture, enabling flexible integration of additional modalities without modifying the core network.
Title and authors: Tom: So, we've talked about how M2Retinexformer takes Retinexformer and adds depth, luminance, and semantic features to tackle low-light images better than just using RGB data alone.
Jane: Right, so the core idea is fusing different kinds of visual information—geometry from depth, brightness details from luminance, and context from semantics—to make those dark scenes look much clearer.
Lu: The big mechanism they use is this Multi-Modal Cross-Attention Block, which lets the standard RGB features ask questions to all those auxiliary inputs simultaneously.
Meng: And what makes it interesting for us in terms of deployment is that they have this adaptive gating thing, where the model decides how much to trust each input based on how reliable it thinks that specific feature is right then.
Tom: That adaptive gating sounds like something we need to worry about when we put models out there. It means the system isn't just blindly trusting everything; it's smart about which cue—depth or semantic—is actually helpful for the current image quality issue.
Jane: Exactly, because the paper shows that this fusion helps distinguish between dark areas caused by a shadow, an object blocking light, or just being far away. It uses geometry to help separate those things.
Lu: And they also use luminance and semantic features to keep the colors natural and textures intact even when the image is dark. They treat those as persistent pieces of information throughout the three refinement stages.
Tom: The training setup involves a progressive refinement strategy, running three identical stages and reusing the same features to keep things from getting too slow or memory intensive during training.
Meng: Three stages sounds manageable for compute, but I need to know how much overhead that adds when we move to real traffic video at scale.
Jane: The results on those standard low-light benchmarks show M2Retinexformer beats the previous Retinexformer and other methods in terms of performance across LOL, SID, SMID, and SDSD sets.
Lu: And the ablation study confirmed that depth was a major contributor to those gains specifically when they added that perceptual loss supervision. It’s not just adding data; it's using it correctly with the right loss function.
Tom: So what this means for us in terms of real-world image enhancement is that we can build a more robust pipeline by explicitly feeding in geometry and content alongside the standard color information.
Jane: It suggests that future models should stop treating low-light images as just a problem of brightening the pixels, and start treating them as a problem of reconstructing scene structure using all available context.
Lu: The limitation they flag is that this entire multi-modal benefit hinges on those auxiliary features being reliable; if the depth estimation fails or the semantic features get noisy, the whole system degrades.
Meng: That makes sense. If we deploy this to a situation where depth sensors are unreliable, we’re back to square one, so we need a way to handle that instability better than just adding more inputs.
Tom: So, M2Retinexformer gives us a solid blueprint for how to build multi-modal enhancement systems by fusing geometry, light cues, and scene context intelligently through adaptive attention.
Jane: It’s definitely a step toward making image restoration less brittle when dealing with the challenging conditions of low-light environments.
Lu: It shows that representation learning from multiple sources is a very effective way to capture the underlying structure needed for high-fidelity restoration in these tough conditions.
The paper's summary: Tom: So, we've seen how M2Retinexformer uses depth and other features to improve low-light images using a multi-modal cross-attention block for better results on those specific benchmarks.
Jane: And now we’re looking at what they actually suggest is the big improvement in the system itself, beyond just getting a higher score on some numbers.
Lu: The key suggestion here is that this framework provides a way to handle scene understanding by fusing geometric context from depth with content guidance from luminance and semantic features.
Meng: So, instead of just trying to make the pixels brighter, we’re looking at rebuilding the representation of the entire scene using these different data streams.
Tom: That means for someone in production, it suggests a pipeline that can be more robust because it's not relying on just one type of visual cue to fix a problem.
Jane: It also talks about how they use an adaptive gating mechanism that lets the model dynamically decide which input—is it the depth map or the semantic features—it should lean on at any given moment based on what looks most trustworthy.
Lu: That’s important because it acknowledges that not every auxiliary feature is useful everywhere; if a specific modality becomes unstable, the system can automatically down-weight its contribution.
Meng: I like that idea of dynamic weighting because when you're deploying this, you don't want a single bad sensor or a noisy data stream to crash the whole enhancement process.
Tom: And they show that by using a combined loss function—the L1 loss and the perceptual loss—they really force the model to preserve those fine textures and high-level structures instead of just getting blurry, bright results.
Jane: That means for a user, it translates to images that don't just look brighter but also retain the natural look of things, like keeping colors consistent in a complex scene.
Lu: The implication is that this moves low-light enhancement from being purely a pixel-level problem to being a richer reconstruction problem where geometry and context are part of the core output.
Tom: It’s about building representations that capture more meaning about the physical world, not just the light intensity on a sensor.
Jane: So, what this changes for someone just listening is that we can expect future AI systems to think more like humans when they encounter things in total darkness—understanding not just what it looks like, but where things are and what they are made of.
The paper's improvements: Tom: So, to wrap up on M2Retinexformer: it’s about taking that existing Retinexformer framework and making it much more powerful by fusing depth, luminance, and semantic features through a smart cross-attention block.
Jane: Right, the main thing is showing how combining these different data types—geometry and context—gives us a more complete picture of what’s happening in a dark scene.
Lu: It moves the enhancement from just brightening pixels to reconstructing the whole scene using multiple contextual cues simultaneously, which is ambitious.
Meng: From my side, it's interesting that they built this modular design so we can potentially swap out or add new data streams later without having to completely rewrite the core enhancement logic.
Lalam: I’m confident that by dynamically weighing the reliability of each modality through that adaptive gating, the model is making a very sure decision about which piece of information to use right now.
Tom: It means for image processing, we’re looking at models that are less likely to fail spectacularly when the input data is messy because they can intelligently ignore noisy inputs.
Jane: That sounds like a big step toward more reliable AI applications in real-world settings where the conditions aren't perfect.
Lu: The limitation they mentioned is that all this benefit depends on those auxiliary features being actually stable; if you feed it bad depth data, the whole multi-modal advantage disappears.
Meng: I agree, because running models at scale means dealing with unpredictable inputs constantly, and instability is a huge deployment headache.
Tom: So we’re left with a model that performs better on LOL, SID, SMID, and SDSD benchmarks because it learned how to use its depth and semantic maps effectively.
Jane: It shows us that low-light enhancement isn't just about a single trick; it requires synthesizing geometric structure with high-level scene meaning.
Lu: The big picture is that we’re moving toward systems where the AI understands not just light, but also the physical layout and context of the environment around it.
Tom: So, M2Retinexformer proves that combining geometry and content guidance through attention layers leads to a solid lift over single-modality models.
Jane: It’s a solid step toward making image restoration less brittle when dealing with the challenging conditions of low-light environments.
Conclusion: Tom: So we’ve looked at M2Retinexformer, which extends Retinexformer by fusing depth, luminance, and semantic features through an adaptive cross-attention block for better low-light results on the LOL, SID, SMID and SDSD benchmarks.
Jane: Right. The core idea is taking that existing Retinexformer and adding those extra ingredients—depth cues for geometry, luminance for local detail, and semantic features for scene context—to get much clearer images in dark scenes than just using standard RGB data alone.
Lu: The big mechanism they use is this Multi-Modal Cross-Attention Block that lets the regular RGB features ask questions to all those extra inputs at the same time.
Meng: And what’s interesting for us as engineers is this adaptive gating thing, where the model decides how much to trust each input based on how reliable it thinks that specific feature is right then.
Tom: That sounds like something we have to watch closely when we put models out there, because it means the system isn't just blindly trusting everything; it's smart about which cue—is it depth or semantic—is actually helpful for the current image quality issue.
Jane: Exactly, because they show that this fusion helps distinguish between dark areas caused by a shadow, an object blocking light, or just being far away; it uses geometry to help separate those things.
Lu: And they also use luminance and semantic features to keep the colors natural and textures intact even when the image is dark; they treat those as persistent pieces of information throughout the three refinement stages.
Tom: The training setup involves a progressive refinement strategy where they run three identical stages and reuse those modality features across them to keep things from getting too slow or memory intensive during training.
Meng: Three stages sounds manageable for compute, but I still need to see some benchmarks on real traffic data before I can say this is ready for production deployment at scale.
Jane: The results on those standard low-light benchmarks show M2Retinexformer beats the previous Retinexformer and other methods in terms of performance across LOL, SID, SMID and SDSD sets.
Lu: And the ablation study on LOLv2 Real confirmed that depth provided the most significant performance gains specifically when they used that perceptual loss supervision; it’s not just adding data; it's using it correctly with the right loss function.
Tom: So what this means for us in terms of real-world image enhancement is that we can build a more robust pipeline by explicitly feeding in geometry and content alongside the standard color information.
Jane: It suggests that future models should stop treating low-light images as just a problem of brightening the pixels and start treating them as a problem of reconstructing scene structure using all available context.
Lu: The limitation they flag is that this entire multi-modal benefit hinges on those auxiliary features being reliable; if the depth estimation fails or the semantic features get noisy, the whole system degrades.
Meng: That makes sense. If we deploy this to a situation where depth sensors are unreliable, we’re back to square one, so we need a way to handle that instability better than just adding more inputs.
Tom: So M2Retinexformer gives us a solid blueprint for how to build multi-modal enhancement systems by fusing geometry, light cues and scene context intelligently through adaptive attention.
Jane: It’s definitely a step toward making image restoration less brittle when dealing with the challenging conditions of low-light environments.
Lu: It shows that representation learning from multiple sources is a very effective way to capture the underlying structure needed for high-fidelity restoration in these tough conditions.
Meng: I still need to see some benchmarks on real traffic data before I can say this is ready for production deployment at scale.
Lalam: Based on the literature I'm seeing, this approach of using cross-attention to selectively weigh different information sources is a very robust way to handle uncertainty in image enhancement tasks; it’s quite confident in its proposed mechanism.
Tom: Yeah, we’ll keep an eye on those deployment numbers while we look at how other models handle the uncertainty quantification we talked about earlier.
Alexandria University
cs.CV
Submitted: 2026-05-11
Updated: 2026-05-11
Comments: Accepted at 2026 IEEE International Conference on Image Processing (ICIP)
DOI: 10.1109/ICIP61757.2026.11630376
Code: https://github.com/YoussefAboelwafa/M2Retinexformer
License: http://creativecommons.org/licenses/by/4.0/
Importance score: 79/100
The gist: M2Retinexformer proposes a novel framework that extends Retinexformer by incorporating depth cues, luminance priors, and semantic features within a progressive refinement pipeline.
Key concepts
- Multi-Modal Cross-Attention Block (MMCAB)
- This is the core fusion module that combines RGB image features with auxiliary data like depth and semantics. It reshapes these different feature types into tokens so that the main image features can selectively query complementary information from the auxiliary modalities, ensuring a rich, integrated representation for enhancement.
- Depth Cues
- Depth maps provide geometric structure to an image, which is independent of lighting changes. This allows the model to understand spatial relationships and disambiguate dark areas that might be caused by shadows or occlusions based on physical distance rather than just pixel intensity.
- Adaptive Gating Mechanism
- This mechanism dynamically decides how much weight to give to different types of information, balancing self-attention (looking at the image itself) and cross-attention (looking at auxiliary data). It adjusts this balance based on how reliable each auxiliary modality is, ensuring the enhancement process uses only the most effective cues.
Terminology
Summary
M2Retinexformer proposes a novel framework that extends Retinexformer by incorporating depth cues, luminance priors, and semantic features within a progressive refinement pipeline. p. 0 While Retinex-based deep learning
methods have achieved promising results, they primarily
rely on single-modality RGB information. We propose
M2Retinexformer (Multi-Modal Retinexformer), a novel
framework that extends Retinexformer by incorporating depth
cues, luminance priors, and semantic features within a progressive refinement pipeline.Where the paper says it
The core innovation of the proposed method involves multi-modal fusion for low-light image enhancement. p. 4 A limitation is that the benefits of multi-modal fusion
depend on modality reliability, and gains may diminish when
auxiliary features are unstable. The proposed framework further provides a modular and extensible design that can accommodate additional priors, making it a promising direction for
future advances in low-light image enhancement. The depth maps remain consistent under different illumination conditions, demonstrating that depth estimation is largely independent of image brightness. p. 1 (a) Normal-light image depth
(b) Low-light image depth
Fig. The depth maps remain consistent under different illumination conditions, demonstrating that depth estimation is largely independent of image brightness. • We introduce M2Retinexformer that extends Retinexformer [5] by incorporating depth, luminance, and
semantic features as auxiliary modalities through a
Multi-Modal Cross-Attention Block (MMCAB) and an
adaptive gating mechanism that balances self-attention
and cross-attention based on auxiliary reliability. The
proposed design fuses heterogeneous modality features
within a modular and extensible architecture, enabling
flexible integration of additional modalities without
modifying the core network.Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Modality Extraction and Feature Engineering
The Modality Extractor is designed to overcome the limitations of RGB-only enhancement by incorporating complementary auxiliary modalities. The paper incorporates three key auxiliary modalities:
-
Depth, which provides illumination-invariant geometric structure that helps disambiguate dark regions caused by shadows, occlusions, or distance. p. 2
Where the paper says it
Modality Extractor To overcome the limitations of RGB-only enhancement, we incorporate complementary auxiliary modalities such as: (i) Depth. Depth provides illumination-invariant geometric structure that helps disambiguate dark regions caused by shadows, occlusions, or distance.
-
Luminance, which is augmented using NTSC luminance enriched with Sobel edges, local contrast, and multi-scale pyramid cues from the same input. p. 2
Where the paper says it
Augmented luminance uses NTSC luminance, L = 0. 114IB, where IR, IG, and IB are the RGB channels enriched with Sobel edges, local contrast, and multi-scale pyramid cues from the same input.
-
Semantic Features extracted using a frozen DINOv3 backbone to provide high-level contextual guidance that helps preserve color consistency and structural integrity in semantically complex regions. p. 2
Where the paper says it
To provide high-level contextual guidance, we extract semantic features using a frozen DINOv3 [26] backbone, which captures object-aware representations that help preserve color consistency and structural integrity in semantically complex regions. For each modality m, features are extracted at multiple scales s ∈ {0, 1, 2} and projected into a unified feature representation F s m ∈ R H 2s × W 2s ×2 sC aligned with Fin.
The Fusion Mechanism
The Multi-Modal Cross-Attention Block (MMCAB) is the core fusion module that integrates RGB features with auxiliary modalities via cross-attention. The MMCAB reshapes RGB and modality features into tokens, where queries are derived from RGB features, while keys and values are obtained from the auxiliary modality. p. 2 Given RGB features Fin and
modality features F
s
m at scale s, we reshape them into tokens
X, Xm ∈ R
N×C
′
with N = H′W′
. Queries are derived
from RGB features, while keys and values are obtained from
the auxiliary modality:
Q = XWQ, Km = XmWKm, Vm = XmWVm, (2) with Q, Km, Vm ∈ R
N×C
′
, and WQ, WKm, WVm ∈ R
C
′×C
′
are learnable projection matrices. The resulting crossattention Am ∈ R
N×C
′
for modality m is computed as:
Am = softmax
QK⊤
√ m
C′
Vm, (3)
allowing RGB features to selectively query complementary
information from auxiliary modalities. Adaptive Gating. Cross-attention output for each modality
Am is weighted by a learnable gate gm based on its reliability:
U =
X
m
gm ⊙ Am, gm = σ(WmX + bm), (6)
This multi-modal output U ∈ R
N×C
′
is then combined
with the self-attention output S via a final gate gf that balances illumination-guided self-attention with multi-modal
cross-attention:
Output = gf ⊙S+(1−gf )⊙U, gf = σ(WfX +bf ), (7)
where Wm, Wf , bm and bf are learnable.Where the paper says it
Where the paper says it
Where the paper says it
Progressive Refinement and Loss Function
The network employs a Progressive Refinement strategy where it cascades three identical refinement stages, reusing modality features across stages to reduce computational overhead. The loss function is a combined objective consisting of the L1 loss and a perceptual loss. p. 3 We find
that incorporating a perceptual loss [28] improves visual
quality and preserves high-level semantic structures and textures that are relevant for low-light enhancement, where fine
details can easily be lost during brightness adjustment. The
combined objective is:
L = L1 + λperLper, (9)
where L1 = ∥Ien − Igt∥1 is the L1 loss between Ien and the
ground truth image. We find
that incorporating a perceptual loss [28] improves visual
quality and preserves high-level semantic structures and textures that are relevant for low-light enhancement, where fine
details can easily be lost during brightness adjustment. The
combined objective is:
L = L1 + λperLper, (9)
where L1 = ∥Ien − Igt∥1 is the L1 loss between Ien and the
ground truth image. We
set λper = 0. 5 based on validation performance. Loss Function
Retinexformer originally employed only the L1 loss. We find
that incorporating a perceptual loss [28] improves visual
quality and preserves high-level semantic structures and textures that are relevant for low-light enhancement, where fine
details can easily be lost during brightness adjustment.Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Performance Evaluation
Evaluations on the LOL, SID, SMID, and SDSD benchmarks demonstrate overall improvements over Retinexformer and recent state-of-the-art methods. Quantitative results show that M2Retinexformer achieves the best or second-best performance on most benchmarks. p. 3 Table 1 compares our method with several recent approaches. M2Retinexformer achieves the best or
second-best performance on most benchmarks, demonstrating the robustness and applicability of the proposed architecture, as well as the effectiveness of the multi-modal fusion and Ablation Study
We conducted a comprehensive ablation study on the LOLv2 Real dataset to quantify each component’s contribution
and validate our design choices. As shown in Table 2, under
perceptual loss supervision, depth yields the most significant
performance gains, followed by luminance. Evaluations across multiple benchmarks show that our
model provides overall performance gains over existing methods. A limitation is that the benefits of multi-modal fusion
depend on modality reliability, and gains may diminish when
auxiliary features are unstable. A limitation is that the benefits of multi-modal fusion
depend on modality reliability, and gains may diminish when
auxiliary features are unstable. The proposed framework further provides a modular and extensible design that can accommodate additional priors, making it a promising direction for
future advances in low-light image enhancement.Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
References
[1] Yuen Peng Loh and Chee Seng Chan, “Getting to know lowlight images with the exclusively dark dataset,” Computer vision and image understanding, 2019.
[2] Edwin H Land and John J McCann, “Lightness and retinex theory,” Journal of the Optical society of America, 1971. p. 5 REFERENCES
[1] Yuen Peng Loh and Chee Seng Chan, “Getting to know lowlight images with the exclusively dark dataset,” Computer vision and image understanding, 2019. [2] Edwin H Land and John J McCann, “Lightness and retinex
theory,” Journal of the Optical society of America, 1971.Where the paper says it
[3] Chen Wei, Wenjing Wang, Wenhan Yang, and Jiaying Liu, “Deep retinex decomposition for low-light enhancement,” arXiv preprint arXiv:1808. p. 5 [2] Edwin H Land and John J McCann, “Lightness and retinex
theory,” Journal of the Optical society of America, 1971. [3] Chen Wei, Wenjing Wang, Wenhan Yang, and Jiaying Liu,
“Deep retinex decomposition for low-light enhancement,”
arXiv preprint arXiv:1808. [3] Chen Wei, Wenjing Wang, Wenhan Yang, and Jiaying Liu,
“Deep retinex decomposition for low-light enhancement,”
arXiv preprint arXiv:1808. 04560, 2018.Where the paper says it
Where the paper says it
[4] Yonghua Zhang, Jiawan Zhang, and Xiaojie Guo, “Kindling the darkness: A practical low-light image enhancer,” in Proceedings of the 27th ACM international conference on multimedia, 2019. p. 5 04560, 2018. [4] Yonghua Zhang, Jiawan Zhang, and Xiaojie Guo, “Kindling
the darkness: A practical low-light image enhancer,” in Proceedings of the 27th ACM international conference on multimedia, 2019.Where the paper says it
[5] Yuanhao Cai, Hao Bian, Jing Lin, Haoqian Wang, Radu Timofte, and Yulun Zhang, “Retinexformer: One-stage retinexbased transformer for low-light image enhancement,” in ICCV, 2023. p. 5 [4] Yonghua Zhang, Jiawan Zhang, and Xiaojie Guo, “Kindling
the darkness: A practical low-light image enhancer,” in Proceedings of the 27th ACM international conference on multimedia, 2019. [5] Yuanhao Cai, Hao Bian, Jing Lin, Haoqian Wang, Radu Timofte, and Yulun Zhang, “Retinexformer: One-stage retinexbased transformer for low-light image enhancement,” in ICCV,
2023.Where the paper says it
[6] Jiesong Bai, Yuhao Yin, Qiyuan He, Yuanxian Li, and Xiaofeng Zhang, “Retinexmamba: Retinex-based mamba for low-light image enhancement,” in International Conference on Neural Information Processing. p. 5 [5] Yuanhao Cai, Hao Bian, Jing Lin, Haoqian Wang, Radu Timofte, and Yulun Zhang, “Retinexformer: One-stage retinexbased transformer for low-light image enhancement,” in ICCV,
2023. [6] Jiesong Bai, Yuhao Yin, Qiyuan He, Yuanxian Li, and Xiaofeng Zhang, “Retinexmamba: Retinex-based mamba for
low-light image enhancement,” in International Conference
on Neural Information Processing. [6] Jiesong Bai, Yuhao Yin, Qiyuan He, Yuanxian Li, and Xiaofeng Zhang, “Retinexmamba: Retinex-based mamba for
low-light image enhancement,” in International Conference
on Neural Information Processing. Springer, 2024.Where the paper says it
Where the paper says it
[7] Alexandru Brateanu, Raul Balmez, Ciprian Orhei, Codruta Ancuti, and Cosmin Ancuti, “Modalformer: Multimodal transformer for low-light image enhancement,” arXiv preprint arXiv:2507. p. 5 Springer, 2024. [7] Alexandru Brateanu, Raul Balmez, Ciprian Orhei, Codruta Ancuti, and Cosmin Ancuti, “Modalformer: Multimodal transformer for low-light image enhancement,” arXiv preprint
arXiv:2507. [7] Alexandru Brateanu, Raul Balmez, Ciprian Orhei, Codruta Ancuti, and Cosmin Ancuti, “Modalformer: Multimodal transformer for low-light image enhancement,” arXiv preprint
arXiv:2507. 20388, 2025.Where the paper says it
Where the paper says it
[8] Glenn Hines, Zia-ur Rahman, Daniel Jobson, and Glenn Woodell, “Single-scale retinex using digital signal processors,” in Global Signal Processing Conference, 2005. p. 5 20388, 2025. [8] Glenn Hines, Zia-ur Rahman, Daniel Jobson, and Glenn Woodell, “Single-scale retinex using digital signal processors,” in
Global Signal Processing Conference, 2005.Where the paper says it
[9] Ana Belen Petro, Catalina Sbert, and Jean-Michel Morel, “Multiscale retinex,” Image processing on line, 2014. p. 5 [8] Glenn Hines, Zia-ur Rahman, Daniel Jobson, and Glenn Woodell, “Single-scale retinex using digital signal processors,” in
Global Signal Processing Conference, 2005. [9] Ana Belen Petro, Catalina Sbert, and Jean-Michel Morel, ´
“Multiscale retinex,” Image processing on line, 2014.Where the paper says it
[10] Xiaojie Guo, Yu Li, and Haibin Ling, “Lime: Low-light image enhancement via illumination map estimation,” IEEE Transactions on image processing, 2016. p. 5 [9] Ana Belen Petro, Catalina Sbert, and Jean-Michel Morel, ´
“Multiscale retinex,” Image processing on line, 2014. [10] Xiaojie Guo, Yu Li, and Haibin Ling, “Lime: Low-light image
enhancement via illumination map estimation,” IEEE Transactions on image processing, 2016.Where the paper says it
[11] Chunle Guo, Chongyi Li, Jichang Guo, Chen Change Loy, Junhui Hou, Sam Kwong, and Runmin Cong, “Zero-reference deep curve estimation for low-light image enhancement,” in CVPR, 2020. p. 5 [10] Xiaojie Guo, Yu Li, and Haibin Ling, “Lime: Low-light image
enhancement via illumination map estimation,” IEEE Transactions on image processing, 2016. [11] Chunle Guo, Chongyi Li, Jichang Guo, Chen Change Loy, Junhui Hou, Sam Kwong, and Runmin Cong, “Zero-reference
deep curve estimation for low-light image enhancement,” in
CVPR, 2020.Where the paper says it
[12] Mariam Saeed and Marwan Torki, “Lit the darkness: Threestage zero-shot learning for low-light enhancement with multineighbor enhancement factors,” in ICASSP. p. 5 [11] Chunle Guo, Chongyi Li, Jichang Guo, Chen Change Loy, Junhui Hou, Sam Kwong, and Runmin Cong, “Zero-reference
deep curve estimation for low-light image enhancement,” in
CVPR, 2020. [12] Mariam Saeed and Marwan Torki, “Lit the darkness: Threestage zero-shot learning for low-light enhancement with multineighbor enhancement factors,” in ICASSP. [12] Mariam Saeed and Marwan Torki, “Lit the darkness: Threestage zero-shot learning for low-light enhancement with multineighbor enhancement factors,” in ICASSP. IEEE, 2023.Where the paper says it
Where the paper says it
[13] Wenhui Wu, Jian Weng, Pingping Zhang, Xu Wang, Wenhan Yang, and Jianmin Jiang, “Uretinex-net: Retinex-based deep unfolding network for low-light image enhancement,” in CVPR, 2022. p. 5 IEEE, 2023. [13] Wenhui Wu, Jian Weng, Pingping Zhang, Xu Wang, Wenhan Yang, and Jianmin Jiang, “Uretinex-net: Retinex-based
deep unfolding network for low-light image enhancement,” in
CVPR, 2022.Where the paper says it
[14] Syed Waqas Zamir, Aditya Arora, Salman Khan, Munawar Hayat, Fahad Shahbaz Khan, and Ming-Hsuan Yang, “Restormer: Efficient transformer for high-resolution image restoration,” in CVPR, 2022. p. 5 [13] Wenhui Wu, Jian Weng, Pingping Zhang, Xu Wang, Wenhan Yang, and Jianmin Jiang, “Uretinex-net: Retinex-based
deep unfolding network for low-light image enhancement,” in
CVPR, 2022. [14] Syed Waqas Zamir, Aditya Arora, Salman Khan, Munawar Hayat, Fahad Shahbaz Khan, and Ming-Hsuan Yang,
“Restormer: Efficient transformer for high-resolution image
restoration,” in CVPR, 2022.Where the paper says it
[15] Zhendong Wang, Xiaodong Cun, Jianmin Bao, Wengang Zhou, Jianzhuang Liu, and Houqiang Li, “Uformer: A general u-shaped transformer for image restoration,” in CVPR, 2022. p. 5 [14] Syed Waqas Zamir, Aditya Arora, Salman Khan, Munawar Hayat, Fahad Shahbaz Khan, and Ming-Hsuan Yang,
“Restormer: Efficient transformer for high-resolution image
restoration,” in CVPR, 2022. [15] Zhendong Wang, Xiaodong Cun, Jianmin Bao, Wengang
Zhou, Jianzhuang Liu, and Houqiang Li, “Uformer: A general
u-shaped transformer for image restoration,” in CVPR, 2022.Where the paper says it
[16] Xiaogang Xu, Ruixing Wang, Chi-Wing Fu, and Jiaya Jia, “Snr-aware low-light image enhancement,” in CVPR, 2022. p. 5 [15] Zhendong Wang, Xiaodong Cun, Jianmin Bao, Wengang
Zhou, Jianzhuang Liu, and Houqiang Li, “Uformer: A general
u-shaped transformer for image restoration,” in CVPR, 2022. [16] Xiaogang Xu, Ruixing Wang, Chi-Wing Fu, and Jiaya Jia,
“Snr-aware low-light image enhancement,” in CVPR, 2022.Where the paper says it
[17] Song Liu, Hongying Zhang, Xue Li, and Xi Yang, “Retinexformer+: Retinex-based dual-channel transformer for low-light image enhancement. p. 5 [16] Xiaogang Xu, Ruixing Wang, Chi-Wing Fu, and Jiaya Jia,
“Snr-aware low-light image enhancement,” in CVPR, 2022. [17] Song Liu, Hongying Zhang, Xue Li, and Xi Yang, “Retinexformer+: Retinex-based dual-channel transformer for low-light
image enhancement. [17] Song Liu, Hongying Zhang, Xue Li, and Xi Yang, “Retinexformer+: Retinex-based dual-channel transformer for low-light
image enhancement. ,” Computers, Materials & Continua,
2025.Where the paper says it
Where the paper says it
[18] Albert Gu and Tri Dao, “Mamba: Linear-time sequence modeling with selective state spaces,” in First conference on language modeling, 2024. p. 5 ,” Computers, Materials & Continua,
2025. [18] Albert Gu and Tri Dao, “Mamba: Linear-time sequence modeling with selective state spaces,” in First conference on language modeling, 2024.Where the paper says it
[19] Chunming He, Chengyu Fang, Yulun Zhang, Longxiang Tang, Jinfa Huang, Kai Li, Xiu Li, Sina Farsiu, et al. p. 5 [18] Albert Gu and Tri Dao, “Mamba: Linear-time sequence modeling with selective state spaces,” in First conference on language modeling, 2024. [19] Chunming He, Chengyu Fang, Yulun Zhang, Longxiang Tang,
Jinfa Huang, Kai Li, Xiu Li, Sina Farsiu, et al. [19] Chunming He, Chengyu Fang, Yulun Zhang, Longxiang Tang,
Jinfa Huang, Kai Li, Xiu Li, Sina Farsiu, et al. , “Reti-diff:
Illumination degradation image restoration with retinex-based
latent diffusion model,” in ICLR, 2025.Where the paper says it
Where the paper says it
[20] Hossam Elkordi, Hicham G Elmongui, and Marwan Torki, “Pwc-diff: Pixel-weighted conditional diffusion for low-light image enhancement,” in ISCC, 2026. p. 5 , “Reti-diff:
Illumination degradation image restoration with retinex-based
latent diffusion model,” in ICLR, 2025. [20] Hossam Elkordi, Hicham G Elmongui, and Marwan Torki,
“Pwc-diff: Pixel-weighted conditional diffusion for low-light
image enhancement,” in ISCC, 2026.Where the paper says it
[21] Zhen Wang, Dongyuan Li, Guang Li, Ziqing Zhang, and Renhe Jiang, “Multimodal low-light image enhancement with depth information,” in Proceedings of the 32nd ACM International Conference on Multimedia, 2024. p. 5 [20] Hossam Elkordi, Hicham G Elmongui, and Marwan Torki,
“Pwc-diff: Pixel-weighted conditional diffusion for low-light
image enhancement,” in ISCC, 2026. [21] Zhen Wang, Dongyuan Li, Guang Li, Ziqing Zhang, and Renhe
Jiang, “Multimodal low-light image enhancement with depth
information,” in Proceedings of the 32nd ACM International
Conference on Multimedia, 2024.Where the paper says it
[22] Pingping Liu, Xiaoyi Wang, Tongshun Zhang, and Liyuan Yin, “Multi-modal fusion guided retinex-based low-light image enhancement,” Expert Systems with Applications, 2025. p. 5 [21] Zhen Wang, Dongyuan Li, Guang Li, Ziqing Zhang, and Renhe
Jiang, “Multimodal low-light image enhancement with depth
information,” in Proceedings of the 32nd ACM International
Conference on Multimedia, 2024. [22] Pingping Liu, Xiaoyi Wang, Tongshun Zhang, and Liyuan Yin,
“Multi-modal fusion guided retinex-based low-light image enhancement,” Expert Systems with Applications, 2025.Where the paper says it
[23] Zhen Wang, Yaozu Wu, Dongyuan Li, Shiyin Tan, and Zhishuai Yin, “Thermal-aware low-light image enhancement: A real-world benchmark and a new light-weight model,” in Proceedings of the AAAI Conference on Artificial Intelligence, 2025. p. 5 [22] Pingping Liu, Xiaoyi Wang, Tongshun Zhang, and Liyuan Yin,
“Multi-modal fusion guided retinex-based low-light image enhancement,” Expert Systems with Applications, 2025. [23] Zhen Wang, Yaozu Wu, Dongyuan Li, Shiyin Tan, and
Zhishuai Yin, “Thermal-aware low-light image enhancement:
A real-world benchmark and a new light-weight model,” in
Proceedings of the AAAI Conference on Artificial Intelligence,
2025.Where the paper says it
[24] Roman Bachmann, Oguzhan F Kar, David Mizrahi, Ali Gar- ˘jani, Mingfei Gao, David Griffiths, Jiaming Hu, Afshin Dehghan, and Amir Zamir, “4m-21: An any-to-any vision model for tens of tasks and modalities,” Advances in Neural Information Processing Systems, 2024. p. 5 [23] Zhen Wang, Yaozu Wu, Dongyuan Li, Shiyin Tan, and
Zhishuai Yin, “Thermal-aware low-light image enhancement:
A real-world benchmark and a new light-weight model,” in
Proceedings of the AAAI Conference on Artificial Intelligence,
2025. [24] Roman Bachmann, Oguzhan F Kar, David Mizrahi, Ali Gar- ˘
jani, Mingfei Gao, David Griffiths, Jiaming Hu, Afshin Dehghan, and Amir Zamir, “4m-21: An any-to-any vision model
for tens of tasks and modalities,” Advances in Neural Information Processing Systems, 2024.Where the paper says it
[25] Lihe Yang, Bingyi Kang, Zilong Huang, Zhen Zhao, Xiaogang Xu, Jiashi Feng, and Hengshuang Zhao, “Depth anything v2,” Advances in Neural Information Processing Systems, 2024. p. 5 [24] Roman Bachmann, Oguzhan F Kar, David Mizrahi, Ali Gar- ˘
jani, Mingfei Gao, David Griffiths, Jiaming Hu, Afshin Dehghan, and Amir Zamir, “4m-21: An any-to-any vision model
for tens of tasks and modalities,” Advances in Neural Information Processing Systems, 2024. [25] Lihe Yang, Bingyi Kang, Zilong Huang, Zhen Zhao, Xiaogang
Xu, Jiashi Feng, and Hengshuang Zhao, “Depth anything v2,”
Advances in Neural Information Processing Systems, 2024.Where the paper says it
[26] Oriane Simeoni, Huy V Vo, Maximilian Seitzer, Federico Baldassarre, Maxime Oquab, Cijo Jose, Vasil Khalidov, Marc Szafraniec, Seungeun Yi, Michael Ramamonjisoa, et al. p. 5 [25] Lihe Yang, Bingyi Kang, Zilong Huang, Zhen Zhao, Xiaogang
Xu, Jiashi Feng, and Hengshuang Zhao, “Depth anything v2,”
Advances in Neural Information Processing Systems, 2024. [26] Oriane Simeoni, Huy V Vo, Maximilian Seitzer, Federico Bal- ´
dassarre, Maxime Oquab, Cijo Jose, Vasil Khalidov, Marc
Szafraniec, Seungeun Yi, Michael Ramamonjisoa, et al. [26] Oriane Simeoni, Huy V Vo, Maximilian Seitzer, Federico Bal- ´
dassarre, Maxime Oquab, Cijo Jose, Vasil Khalidov, Marc
Szafraniec, Seungeun Yi, Michael Ramamonjisoa, et al. , “Di- ¨
nov3,” arXiv preprint arXiv:2508. , “Di- ¨
nov3,” arXiv preprint arXiv:2508. 10104, 2025.Where the paper says it
Where the paper says it
Where the paper says it
[27] Syed Waqas Zamir, Aditya Arora, Salman Khan, Munawar Hayat, Fahad Shahbaz Khan, Ming-Hsuan Yang, and Ling Shao, “Learning enriched features for fast image restoration and enhancement,” TPAMI, 2022. p. 5 10104, 2025. [27] Syed Waqas Zamir, Aditya Arora, Salman Khan, Munawar
Hayat, Fahad Shahbaz Khan, Ming-Hsuan Yang, and Ling
Shao, “Learning enriched features for fast image restoration
and enhancement,” TPAMI, 2022.Where the paper says it
[28] Justin Johnson, Alexandre Alahi, and Li Fei-Fei, “Perceptual losses for real-time style transfer and super-resolution,” in ECCV. p. 5 [27] Syed Waqas Zamir, Aditya Arora, Salman Khan, Munawar
Hayat, Fahad Shahbaz Khan, Ming-Hsuan Yang, and Ling
Shao, “Learning enriched features for fast image restoration
and enhancement,” TPAMI, 2022. [28] Justin Johnson, Alexandre Alahi, and Li Fei-Fei, “Perceptual losses for real-time style transfer and super-resolution,” in
ECCV. [28] Justin Johnson, Alexandre Alahi, and Li Fei-Fei, “Perceptual losses for real-time style transfer and super-resolution,” in
ECCV. Springer, 2016.Where the paper says it
Where the paper says it
[29] Wenhan Yang, Wenjing Wang, Haofeng Huang, Shiqi Wang, and Jiaying Liu, “Sparse gradient regularized deep retinex network for robust low-light image enhancement,” TIP, 2021. p. 5 Springer, 2016. [29] Wenhan Yang, Wenjing Wang, Haofeng Huang, Shiqi Wang,
and Jiaying Liu, “Sparse gradient regularized deep retinex network for robust low-light image enhancement,” TIP, 2021.Where the paper says it
[30] Chen Chen, Qifeng Chen, Minh N Do, and Vladlen Koltun, “Seeing motion in the dark,” in ICCV, 2019. p. 5 [29] Wenhan Yang, Wenjing Wang, Haofeng Huang, Shiqi Wang,
and Jiaying Liu, “Sparse gradient regularized deep retinex network for robust low-light image enhancement,” TIP, 2021. [30] Chen Chen, Qifeng Chen, Minh N Do, and Vladlen Koltun,
“Seeing motion in the dark,” in ICCV, 2019.Where the paper says it
[31] Chen Chen, Qifeng Chen, Jia Xu, and Vladlen Koltun, “Learning to see in the dark,” in CVPR, 2018. p. 5 [30] Chen Chen, Qifeng Chen, Minh N Do, and Vladlen Koltun,
“Seeing motion in the dark,” in ICCV, 2019. [31] Chen Chen, Qifeng Chen, Jia Xu, and Vladlen Koltun, “Learning to see in the dark,” in CVPR, 2018.Where the paper says it
[32] Ruixing Wang, Xiaogang Xu, Chi-Wing Fu, Jiangbo Lu, Bei Yu, and Jiaya Jia, “Seeing dynamic scene in the dark: A highquality video dataset with mechatronic alignment,” in ICCV, 2021. p. 5 [31] Chen Chen, Qifeng Chen, Jia Xu, and Vladlen Koltun, “Learning to see in the dark,” in CVPR, 2018. [32] Ruixing Wang, Xiaogang Xu, Chi-Wing Fu, Jiangbo Lu, Bei
Yu, and Jiaya Jia, “Seeing dynamic scene in the dark: A highquality video dataset with mechatronic alignment,” in ICCV,
2021.Where the paper says it
[33] Youssef Aboelwafa, Hicham G. p. 0 M2RETINEXFORMER: MULTI-MODAL RETINEXFORMER FOR LOW-LIGHT IMAGE
ENHANCEMENT
Youssef Aboelwafa, Hicham G. Elmongui, Marwan Torki
Alexandria University, Egypt
ABSTRACT
Low-light image enhancement is challenging due to
complex degradations, including amplified noise, artifacts,
and color distortion. M2RETINEXFORMER: MULTI-MODAL RETINEXFORMER FOR LOW-LIGHT IMAGE
ENHANCEMENT
Youssef Aboelwafa, Hicham G. Elmongui, Marwan Torki
Alexandria University, Egypt
ABSTRACT
Low-light image enhancement is challenging due to
complex degradations, including amplified noise, artifacts,
and color distortion.Where the paper says it
Where the paper says it
[34] M2RETINEXFORMER: MULTI-MODAL RETINEXFORMER FOR LOW-LIGHT IMAGE ENHANCEMENT. p. 0 M2RETINEXFORMER: MULTI-MODAL RETINEXFORMER FOR LOW-LIGHT IMAGE
ENHANCEMENT
Youssef Aboelwafa, Hicham G. Elmongui, Marwan Torki
Alexandria University, Egypt
ABSTRACT
Low-light image enhancement is challenging due to
complex degradations, including amplified noise, artifacts,
and color distortion.Where the paper says it
[38] Fig. 1. The proposed M2Retinexformer achieves higher PSNR than the baseline Retinexformer on most evaluated datasets. p. 0 39
RetinexFormer
M2RetinexFormer
Fig. The proposed M2Retinexformer achieves higher PSNR than
the baseline Retinexformer on most evaluated datasets.Where the paper says it
[39] Fig. 2. The depth maps remain consistent under different illumination conditions, demonstrating that depth estimation is largely independent of image brightness.
[40] Table 1. p. 3 866
M2Retinexformer 24. 843
Table 1. 843
Table 1. Quantitative comparisons on LOL v1/v2, SID, SMID, and SDSD datasets.Where the paper says it
Where the paper says it
[41] Table 2. p. 3 866
M2Retinexformer 24. 843
Table 1. 50
Table 2. Ablation study on LOL-v2 Real using τ=3 showing the
contribution of each component.Where the paper says it
Where the paper says it
[43] Fig. 4. Visual results on LOL-v2 Real. p. 4 Input Retinexformer M2Retinexformer Ground Truth
Fig. Visual results on LOL-v2 Real.Where the paper says it
[48] M2Retinexformer has 2M trainable params and 48M total params, including frozen DepthAnything-V2 and DINOv3 encoders that do not add optimization complexity. p. 3 Model complexity. M2Retinexformer has 2M trainable
params and 48M total params, including frozen DepthAnything-V2 and DINOv3 encoders that do not add optimization complexity.Where the paper says it
[49] All experiments are conducted on a single NVIDIA RTX 5090 GPU. p. 3 This is about 1/4 of the 4M-21 extractor [24]
(198M params) used in ModalFormer [7]. All experiments
are conducted on a single NVIDIA RTX 5090 GPU.Where the paper says it
Conclusion
In this paper, we propose M2Retinexformer, a multi-modal extension of Retinexformer that incorporates heterogeneous modalities through cross-attention fusion. The key insight is that depth provides geometric context that is robust to illumination changes, while luminance and semantic features provide content-aware guidance. p. 4 CONCLUSION
In this paper, we propose M2Retinexformer, a multi-modal
extension of Retinexformer that incorporates heterogeneous
modalities through cross-attention fusion. Our key insight
is that depth provides geometric context that is robust to illumination changes, while luminance and semantic features
provide content-aware guidance. Integrated through the proposed MMCAB, these modalities improve enhancement quality. Evaluations across multiple benchmarks show that our
model provides overall performance gains over existing methods. Evaluations across multiple benchmarks show that our
model provides overall performance gains over existing methods. A limitation is that the benefits of multi-modal fusion
depend on modality reliability, and gains may diminish when
auxiliary features are unstable. A limitation is that the benefits of multi-modal fusion
depend on modality reliability, and gains may diminish when
auxiliary features are unstable. The proposed framework further provides a modular and extensible design that can accommodate additional priors, making it a promising direction for
future advances in low-light image enhancement.Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.
Improvements for AI systems
-
Multi-modal fusion for enhanced scene understanding: The M2Retinexformer can now distinguish between dark regions caused by distance, occlusion, or shadows because
Depth provides geometric context that is invariant to lighting variations.
-
Content-aware feature preservation: The system can preserve natural colors and fine textures in complex regions because it maintains luminance features as a
persistent modality
and propagates semantic features throughout the network. -
Adaptive fusion mechanism: The model dynamically balances self-attention and cross-attention based on auxiliary cue reliability, allowing it to selectively incorporate only the most effective modalities for enhancement.
Abstract
Low-light image enhancement is challenging due to complex degradations, including amplified noise, artifacts, and color distortion. While Retinex-based deep learning methods have achieved promising results, they primarily rely on single-modality RGB information. We propose M2Retinexformer (Multi-Modal Retinexformer), a novel framework that extends Retinexformer by incorporating depth cues, luminance priors, and semantic features within a progressive refinement pipeline. Depth provides geometric context that is invariant to lighting variations, while luminance and semantic features offer explicit guidance on brightness distribution and scene understanding. Modalities are extracted at multiple scales and fused through cross-attention, with adaptive gating dynamically balancing illumination-guided self-attention and cross-attention based on the reliability of auxiliary cues. Evaluations on the LOL, SID, SMID, and SDSD benchmarks demonstrate overall improvements over Retinexformer and recent state-of-the-art methods. Code and pretrained weights are available at https://github.com/YoussefAboelwafa/M2Retinexformer
Sources
- Deep Retinex Decomposition for Low-Light Enhancement
- ModalFormer: Multimodal Transformer for Low-Light Image Enhancement
- DINOv3
Related papers
- TEA: Text Encoder Alignment for Robust Concept Erasure in Text-to-Image Models
- Do Egocentric Video-Language Models Capture Both Hand- and Object-Centric Cues?
- Aura: Consistent Multi-Subject Video Generation via VLM-Grounded Semantic Alignment
- PVRF: All-in-one Adverse Weather Removal via Prior-modulated and Velocity-constrained Rectified Flow
- DLWM: Diverse Latent World Models for Efficient Multimodal Reasoning
- Out of Sight, Still in Mind: Token Compression for Omni-LLMs