Retinexformer: One-stage Retinex-based Transformer for Low-light Image Enhancement
Listen
Radio episode about this paper
Transcript
Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.
Tom: We're looking at the paper: "Retinexformer: One-stage Retinex-based Transformer for Low-light Image Enhancement".
Title and authors: Tom: So we're looking at this paper, Retinexformer: One-stage Retinex-based Transformer for Low-light Image Enhancement. It basically takes the old Retinex idea and puts a Transformer on top to fix low light images faster than the old convolutional networks which were slow and couldn't see long distance relationships well.
Jane: Right, so we're starting with an idea that estimates illumination first, then uses that to light up the image, and then uses a Transformer to clean up all the noise and artifacts in one single step instead of a multi-stage process.
Lu: The core idea is this One-stage Retinex-based Framework or ORF which first estimates the illumination information and then uses it to light up the low light image, followed by a corruption restorer to suppress noise, artifacts, under-/overexposure, and color distortion in a single training stage <ref:2303.06705#pg2>.
Meng: It sounds like they’re trying to simplify the pipeline significantly. If you have to run several separate steps for illumination estimation and then corruption restoration, that adds complexity and time on the production side, which is a big thing for me when we're shipping models <ref:2303.06705#pg2>.
Lalam: The framework actually has an illumination estimator and a corruption restorer as its main parts <ref:2303.06705#pg2>. The illumination estimator estimates the light-up image, called Ilu, and a light-up feature called Flu <ref:2303.06705#pg2>.
Tom: And then that illumination information is used to light up the low light images before feeding it into the corruption restorer which is this Illumination-Guided Transformer or IGT <ref:2303.06705#pg2>.
Jane: So, what makes this IGT special? It uses those illumination representations to direct how the self attention works, specifically enhancing interactions between regions of different exposure levels <ref:2303.06705#pg2>.
Lu: The key part there is the Illumination-Guided Multi-head Self-Attention or IGMSA which exploits those illumination representations to direct the computation of self attention and enhance interactions between regions of different lighting conditions <ref:2303.06705#pg2>.
Meng: I see a complexity analysis mentioned. They claim that the computational cost for this IGMSA is linear to the spatial size, which is O(IG-MSA) equals two HW C to the power of two divided by k <ref:2303.06705#pg2>. That’s much better than the quadratic complexity of global MSA used in some previous CNN Transformer methods, which is O(G-MSA) equals two times H times W squared times C <ref:2303.06705#pg2>.
Title and authors: Tom: Linear versus quadratic is a huge win for me because it means this model can actually scale better to larger images without immediately crashing our hardware, which is something I worry about when we try to deploy these Transformer architectures <ref:2303.06705#pg2>.
Jane: So, the paper suggests they revise the original Retinex model by adding perturbation terms to the reflectance and illumination for modeling those corruptions <ref:2303.06705#pg2>. This allows their ORF to estimate illumination information and then use it to light up the low light images <ref:2303.06705#pg2>.
Lu: And after lighting things up, they have a corruption restorer which is the IGT that takes the lit-up image, Ilu, as input to restore corruptions and produce the enhanced image, called Ien <ref:2303.06705#pg2>.
Meng: That’s the end-to-end training aspect they emphasize. Training it all in one stage instead of a multi-stage pipeline is a practical improvement for deployment <ref:2303.06705#pg2>.
Tom: The ablation study on the SDSD outdoor dataset showed that combining ORF and IGMSA achieved three point three seven dB improvements over baseline one <ref:2303.06705#pg2>. That’s a solid number for a performance gain <ref:2303.06705#pg2>.
Jane: And when you look at the broader comparison with other state of the art Transformer-based image restoration algorithms, Retinexformer gains by two point seven three to four point two six dB on seven datasets <ref:2303.06705#pg2>. That’s a pretty wide range of improvement across different tests <ref:2303.06705#pg2>.
Lu: Qualitatively, the results show that Retinexformer can effectively remove noise without introducing spots and artifacts, which is what we need for real-world use <ref:2303.06705#pg2>. Quantitatively, they saw significant improvements on thirteen datasets <ref:2303.06705#pg2>.
Meng: The user study scores also showed the highest average result, which suggests it’s not just good on the benchmarks but actually performs well in a practical assessment of quality <ref:2303.06705#pg2>.
Jane: We also have results for low-light object detection on the ExDark dataset, where Retinexformer achieved an average precision score of sixty-six point one, which is zero point five AP higher than the recent best self-supervised method called SCI thirty-seven <ref:2303.06705#pg2>.
Tom: That’s interesting because it shows its value extends beyond just image enhancement into high level vision understanding, like detecting things in the dark <ref:2303.06705#pg2>. They even got the best results for five object categories: bicycle, boat, bottle, cat, and dog <ref:2303.06705#pg2>.
Lu: That demonstrates practical value in high level vision understanding, which is important because that's where we need robust models <ref:2303.06705#pg2>.
Title and authors: Meng: What I’m curious about from a deployment standpoint is the training signal construction. How did they actually build the training signal for this? Is it something straightforward enough to implement reliably in our pipeline?
Lalam: The ORF consists of an illumination estimator and a corruption restorer as its main parts <ref:2303.06705#pg2>. The illumination estimator estimates the light-up image, called Ilu, and a light-up feature called Flu <ref:2303.06705#pg2>.
Jane: And the corruption restorer, which is the IGT, takes that lit-up image Ilu as input to restore corruptions and produce the enhanced image Ien <ref:2303.06705#pg2>. The basic building block of this IGT is called the Illumination-Guided Attention Block or IGAB <ref:2303.06705#pg2>.
Tom: That IGAB is composed of two layer normalizations, an IG-MSA, and a feed forward network or FFN <ref:2303.06705#pg2>. The crucial part here is that the Illumination-Guided Multi-head Self-Attention, or IGMSA, exploits illumination representations to direct the computation of self attention and enhance interactions between regions of different exposure levels <ref:2303.06705#pg2>.
Lu: And for scaling, they show that this IGMSA treats a single channel feature map as a token to compute self attention linearly with respect to the spatial size, which is linear complexity equals two H times W C to the power of two divided by k <ref:2303.06705#pg2>.
Meng: That linear scaling is what makes this viable for us on larger images, which is exactly why I was asking about it earlier <ref:2303.06705#pg2>. It moves us away from that quadratic complexity we see in other Transformer methods <ref:2303.06705#pg2>.
Tom: So to summarize, the paper Retinexformer is proposing a simple yet principled One-stage Retinex-based Framework or ORF which uses an Illumination-Guided Transformer to model long-range dependencies through illumination representations <ref:2303.06705#pg2>.
Jane: It’s important to remember that they are revising the original Retinex model by introducing perturbation terms to the reflectance and illumination for modeling the corruptions <ref:2303.06705#pg2>. This lets their ORF estimate illumination information and then use it to light up the low light images <ref:2303.06705#pg2>.
Lu: The key part of IGT is that Illumination-Guided Multi-head Self-Attention, or IGMSA, which exploits illumination representations to direct the computation of self attention and enhance interactions between regions of different exposure levels <ref:2303.06705#pg2>.
Meng: From my point of view, we need to be careful about what they don't cover. They focus heavily on enhancement, but I wonder how stable this whole pipeline is when we apply it to very noisy or highly distorted real traffic camera feeds <ref:2303.06705#pg2>.
Title and authors: Tom: They did mention their limitation in the conclusion, that they focused on enhancing low light images and didn't explicitly detail how well it performs on other complex restoration tasks outside of this specific domain <ref:2303.06705#pg2>.
Jane: Exactly, so while the results are impressive across thirteen datasets <ref:2303.06705#pg2>, we should keep an eye on how robust this framework is when dealing with completely different types of corruption that aren't just simple illumination issues <ref:2303.06705#pg2>.
Lu: Overall, Retinexformer is a novel Transformer-based method for low light image enhancement that successfully introduces perturbation terms to the original Retinex model to formulate ORF, and then designs an IGT utilizing illumination information to direct long-range dependencies <ref:2303.06705#pg2>.
Meng: It’s a solid paper because it tackles the long-range dependency issue using a Transformer in a way that keeps the computation manageable for practical application <ref:2303.06705#pg2>.
Tom: The results of this Retinexformer method show that it dramatically outperforms state of the art methods on thirteen datasets <ref:2303.06705#pg2>.
Jane: And the practical values are shown by user study scores and low light detection, where it achieved an average precision score of sixty-six point one on the ExDark dataset, which is zero point five AP higher than the recent best self-supervised method called SCI thirty-seven <ref:2303.06705#pg2>.
Lu: So we’re looking at a method that moves us away from tedious multi-stage pipelines and shows significant gains in both enhancement quality and practical object detection tasks <ref:2303.06705#pg2>.
Meng: It’s an improvement over what we have, especially with the linear complexity of the IGMSA, which is a key scaling factor for us <ref:2303.06705#pg2>.
Tom: It’s a lot to process, but it seems like they’ve managed to make a complex theory computationally efficient enough for real-world use <ref:2303.06705#pg2>.
Jane: In conclusion, Retinexformer is proposed as a novel Transformer-based method for low light image enhancement where the authors successfully introduce perturbation terms to the original Retinex model to formulate ORF, and then design an IGT utilizing illumination information to direct long-range dependencies <ref:2303.06705#pg2>.
Lu: The results of this paper show that Retinexformer dramatically outperforms state of the art methods on thirteen datasets <ref:2303.06705#pg2>.
Meng: And the results of user study and low light detection also demonstrate the practical values of our method <ref:2303.06705#pg2>.
Lalam: So, we have a new way to look at low light restoration using this Retinexformer approach <ref:2303.06705#pg2>.
The paper's summary: Tom: So, to sum up what we just talked about, Retinexformer takes the old Retinex idea and uses a Transformer to fix low light images faster and better than older methods that required several separate steps.
Jane: Exactly. The authors are proposing this One-stage Retinex-based Framework, or ORF. It’s basically their way of doing things all at once instead of having a long pipeline that takes forever to train and run.
Lu: They start by modifying the original Retinex model by adding some new terms for the reflectance and illumination so it can handle those corruptions better, which is how they formulate that ORF <ref:2303.06705#pg2>.
Meng: So, they estimate the illumination first, and then use that light information to illuminate the image, followed by a restorer to clean up all the noise and artifacts in just one training stage <ref:2303.06705#pg2>. That’s what I like about it—one training run instead of three or four.
Tom: Right, and the real muscle here is this Illumination-Guided Transformer, or IGT, which they use to model those long-range dependencies that Transformers are supposed to be good at <ref:2303.06705#pg2>.
Jane: The authors make a really clever point about how the IGT uses that illumination information to guide the self-attention mechanism. It’s like they are telling the attention where to look based on how bright or dark each part of the image is <ref:2303.06705#pg2>.
Lu: The specific component they focus on is this Illumination-Guided Multi-head Self-Attention, or IGMSA. It exploits those illumination representations to direct the attention computation and really enhance how different parts of an image interact, especially when some parts are very dark and others are very bright <ref:2303.06705#pg2>.
Meng: I was looking at the complexity numbers they gave, and this is what caught my eye. They claim that the IGMSA has a linear complexity relative to the spatial size of your image, which is two times H times W times C to the power of two divided by k <ref:2303.06705#pg2>. That’s linear scaling for attention, which is way better than the quadratic complexity of global self-attention we see in some older Transformer methods <ref:2303.06705#pg2>.
Tom: Linear scaling means this thing can actually handle bigger images without immediately blowing up our hardware during inference, which is a massive win for me as an engineer <ref:2303.06705#pg2>.
Jane: And when you look at the actual performance numbers on the SDSD outdoor dataset, they show that by combining the ORF and this IGMSA mechanism, they get improvements of three point three seven decibels over their baseline one <ref:2303.06705#pg2>. That’s a solid improvement for image quality.
Lu: It’s not just about the quality gain, though. When comparing Retinexformer to other state-of-the-art Transformer methods across thirteen different datasets, it gains between two point seven three and four point two six decibels <ref:2303.06705#pg2>.
Meng: That range of improvement is pretty wide, which suggests it’s not just a slight tweak; it’s genuinely better than what we have currently in the Transformer restoration space <ref:2303.06705#pg2>.
Tom: Beyond just image enhancement, they also tested this on low light object detection with the ExDark dataset, and Retinexformer got an average precision score of six point six one <ref:2303.06705#pg2>. That’s half a point higher than the current best self-supervised method, SCI thirty-seven <ref:2303.06705#pg2>.
Jane: So, what this means for someone who just looks at the show—it shows that this isn't just a nice picture filter; it actually helps with real-world vision tasks like identifying objects in the dark <ref:2303.06705#pg2>. It proves the method has practical value beyond just pretty images.
Lu: Overall, Retinexformer is proposing this novel Transformer approach that uses illumination information to guide long-range dependencies, and the evidence on thirteen datasets and in object detection gives me a lot of confidence in its foundation <ref:2303.06705#pg2>.
Meng: It’s a solid piece of work because they managed to solve the long-range dependency problem using a Transformer without making the computation completely unmanageable for deployment <ref:2303.06705#pg2>.
Tom: The paper shows that this approach dramatically outperforms the existing state-of-the-art methods on thirteen datasets, which is what we look for when we're trying to ship a production model <ref:2303.06705#pg2>.
Jane: And those results from the user studies and the object detection scores really show that this method actually performs well in real-world scenarios, not just on some clean benchmark data <ref:2303.06705#pg2>.
Lu: So, we have a new way to look at low light restoration using this Retinexformer approach, moving away from those complicated multi-stage pipelines and achieving great results with efficient scaling <ref:2303.06705#pg2>.
The paper's improvements: Tom: So, we've seen how Retinexformer builds its framework, and now we need to talk about what they actually improved compared to what came before <ref:2303.06705#pg2>.
Jane: Right, so they didn't just build something new; they took the old Retinex idea and made it more principled by introducing these perturbation terms for the reflectance and illumination <ref:2303.06705#pg2>.
Lu: That’s the foundation of their One-stage Retinex-based Framework, or ORF. They use those terms to model the corruptions, which lets them estimate illumination information first <ref:2303.06705#pg2>.
Meng: The next big improvement is how they structure the training itself. Instead of that tedious multi-stage pipeline we talked about before, they trained this entire system end-to-end in a single stage <ref:2303.06705#pg2>. That simplifies deployment significantly for me <ref:2303.06705#pg2>.
Tom: And that leads us back to the Illumination-Guided Transformer, or IGT, which is where the heavy lifting happens in terms of modeling those long-range dependencies <ref:2303.06705#pg2>.
Jane: They show that by using illumination representations to direct the self-attention computation, you can really enhance the interactions between different regions of varying exposure levels <ref:2303.06705#pg2>. It’s like giving the AI a map of where the light is so it knows how to look at things correctly <ref:2303.06705#pg2>.
Lu: The Illumination-Guided Multi-head Self-Attention, or IGMSA, is the core here. It’s what lets them keep that linear complexity with respect to the spatial size of the image rather than going quadratic like global self attention <ref:2303.06705#pg2>. That scaling is a huge technical improvement for large images <ref:2303.06705#pg2>.
Meng: Linear complexity means we can actually run this on high-resolution feeds without immediately hitting memory walls, which is crucial if we're deploying this on edge devices or in real traffic scenarios <ref:2303.06705#pg2>.
Tom: And the results back up the mechanism. The ablation study showed that using both the ORF and this IGMSA together gave us three point three seven decibel improvements over their baseline one <ref:2303.06705#pg2>. That’s a solid number tied directly to their main innovation <ref:2303.06705#pg2>.
Jane: And when we look at the broader comparison with other Transformer methods, Retinexformer gained between two point seven three and four point two six decibels across seven different datasets <ref:2303.06705#pg2>. That shows the effect is pretty consistent <ref:two thousand three hundred three point zero six seven zero five#pg2.
Lu: Qualitatively, they also showed that this method successfully removes noise without creating those distracting spots or artifacts, which is what we need for real-world usability <ref:2303.06705#pg2>.
Meng: The user study scores also showed it performed best on average, which tells me it’s not just good on the numbers but actually delivers better quality in a practical sense for the end user <ref:two thousand three hundred three point zero six seven zero five#pg2.
Tom: So, what this means for us is that we have a method that's not only performing well on standard benchmarks but also showing strong results in real object detection tasks like spotting things in low light on the ExDark dataset <ref:two thousand three hundred three point zero six seven zero five#pg2.
Jane: That’s a really important point because it proves this isn't just an enhancement tool; it’s useful for higher-level vision understanding, which is where we actually need robust models <ref:two thousand three hundred three point zero six seven zero five#pg2.
Lu: The paper flags one limitation, though, and they are pretty clear about it: they focused heavily on enhancing low light images and didn't really detail how stable the whole pipeline is when you try to apply it to completely different types of corruption <ref:two thousand three hundred three point zero six seven zero five#pg2.
Meng: I agree with that, we need to know if this works reliably when the noise isn't just simple illumination issues, like in a messy traffic camera feed <ref:two thousand three hundred three point zero six seven zero five#pg2.
Tom: It’s a lot to process, but it seems they’ve managed to make a complex theory computationally efficient enough for real-world use <ref:two thousand three hundred three point zero six seven zero five#pg2.
Conclusion: Tom: So, to wrap this up, Retinexformer is essentially a novel Transformer approach for low light image enhancement that uses an illumination guide to model those long-range dependencies <ref:2303.06705#pg2>.
Jane: That’s right. They managed to take the original Retinex model, add perturbation terms, and then design this Illumination-Guided Transformer utilizing illumination information for long-range dependencies <ref:2303.06705#pg2>.
Lu: The results show that Retinexformer dramatically outperforms state of the art methods on thirteen datasets <ref:2303.06705#pg2>. It's a lot to take in, but the performance gains are quite clear <ref:two thousand three hundred three point zero six seven zero five#pg2.
Meng: The practical value is shown by the user study scores and low light detection where it hit an average precision score of six point six one on the ExDark dataset <ref:2303.06705#pg2>. That’s a solid number for real-world vision tasks <ref:two thousand three hundred three point zero six seven zero five#pg2.
Lalam: I’m confident in saying that this paper successfully introduces the ORF and then designs the IGT using illumination information to guide those long-range dependencies <ref:2303.06705#pg2>. It’s a very solid method for this specific problem <ref:two thousand three hundred three point zero six seven zero five#pg2.
Tom: It moves us away from those tedious multi-stage pipelines and shows significant gains in both enhancement quality and practical object detection tasks <ref:2303.06705#pg2>.
Jane: Exactly. The numbers on the SDSD dataset show improvements over baseline one, and the comparison with other Transformer methods is pretty strong across seven different tests <ref:two thousand three hundred three point zero six seven zero five#pg2.
Lu: And the linear scaling of that attention mechanism means this framework can actually handle larger images without immediately blowing up our hardware during inference <ref:2303.06705#pg2>. That’s a huge technical win for scalability <ref:two thousand three hundred three point zero six seven zero five#pg2.
Meng: I'm still cautious about how stable this whole pipeline is when you apply it to very noisy or highly distorted real traffic camera feeds, though <ref:2303.06705#pg2>. That’s the main thing we need to watch out for on the production side.
Tom: So, we've seen that Retinexformer is a novel Transformer-based method for low light image enhancement that successfully introduces perturbation terms to the original Retinex model to formulate ORF and then designs an IGT utilizing illumination information <ref:two thousand three hundred three point zero six seven zero five#pg2.
Jane: It’s a very solid method for this specific problem, but we should keep an eye on how robust it is when dealing with completely different types of corruption <ref:two thousand three hundred three point zero six seven zero five#pg2.
Lu: Overall, Retinexformer is a novel Transformer-based method for low light image enhancement that successfully introduces perturbation terms to the original Retinex model to formulate ORF and then designs an IGT utilizing illumination information to direct long-range dependencies <ref:2303.06705#pg2>.
Meng: It’s a solid piece of work because they tackled the long-range dependency issue using a Transformer in a way that keeps the computation manageable for practical application <ref:two thousand three hundred three point zero six seven zero five#pg2.
Tom: The results of this Retinexformer method show that it dramatically outperforms state of the art methods on thirteen datasets <ref:2303.06705#pg2>.
Jane: And the practical values are shown by user study scores and low light detection, where it achieved an average precision score of six point six one on the ExDark dataset, which is half a point higher than the recent best self-supervised method called SCI thirty-seven <ref:two thousand three hundred three point zero six seven zero five#pg2.
Lu: So, we’re looking at a method that moves us away from tedious multi-stage pipelines and shows significant gains in both enhancement quality and practical object detection tasks <ref:two thousand three hundred three point zero six seven zero five#pg2.
Meng: It’s an improvement over what we have, especially with the linear complexity of the IGMSA, which is a key scaling factor for us <ref:two thousand three hundred three point zero six seven zero five#pg2.
Tom: It’s a lot to process, but it seems they’ve managed to make a complex theory computationally efficient enough for real-world use <ref:two thousand three hundred three point zero six seven zero five#pg2.
Yuanhao Cai, Hao Bian, Jing Lin, Haoqian Wang, Radu Timofte, Yulun Zhang
Tsinghua University · University of Wurzburg · ETH Zurich
cs.CV
Submitted: 2023-03-12
Updated: 2026-11-17
Comments: ICCV 2023; The first Transformer-based method for low-light image enhancement
Code: https://github.com/caiyuanhao1998/Retinexformer
License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
Importance score: 90/100
The gist: When enhancing low-light images, many deep learning algorithms are based on the Retinex theory <ref:2303.06705#pg2> but these methods usually require a tedious multi-stage training pipeline and rely…
Key concepts
- One-stage Retinex-based Framework (ORF)
- The ORF revises the original Retinex model by adding perturbation terms for reflectance and illumination. It estimates the illumination first, then uses this estimate to 'light up' the image, followed by a corruption restorer to clean up noise and artifacts in one single training step.
- Illumination-Guided Transformer (IGT)
- The IGT is the core component that models long-range dependencies. It uses illumination representations to direct how self-attention is computed, allowing the model to better understand interactions between different regions of varying exposure levels in the image.
- Illumination-Guided Multi-head Self-Attention (IG-MSA)
- This mechanism treats single illumination channels as tokens for attention. It directs the self-attention computation based on illumination data, which enhances interactions across different exposure levels. This approach achieves linear computational complexity relative to spatial size, unlike quadratic methods.
Terminology
Summary
When enhancing low-light images, many deep learning algorithms are based on the Retinex theory <ref:2303. p. 006705#pg2> but these methods usually require a tedious multi-stage training pipeline and rely on convolutional neural networks, showing limitations in capturing long-range dependencies <ref:2303. p. 006705#pg3>. This paper proposes Retinexformer, a novel Transformer-based method that formulates a simple yet principled One-stage Retinex-based Framework (ORF) which then uses an Illumination-Guided Transformer (IGT) to model long-range dependencies <ref:2303. p. 006705#pg2>.
Pruning redundant reasoning tokens
The authors formulate the ORF by revising the original Retinex model by introducing perturbation terms to the reflectance and illumination for modeling the corruptions <ref:2303.06705#pg2>. The ORF estimates illumination information and then uses it to light up the low-light images, followed by a corruption restorer to suppress noise, artifacts, under-/overexposure, and color distortion <ref:2303. p. 106705#pg2>. This framework is trained end-to-end in a one-stage manner instead of the tedious multi-stage pipeline seen in previous Retinex-based deep learning frameworks <ref:2303. p. 106705#pg2>.
How the training signal is built
The ORF consists of an illumination estimator (i) and a corruption restorer (ii) <ref:2303.06705#pg2>. The illumination estimator estimates the light-up image Ilu and a light-up feature Flu <ref:2303. p. 206705#pg2>. The corruption restorer, which is the IGT, takes the lit-up image Ilu as input to restore corruptions and produce the enhanced image Ien <ref:2303. p. 306705#pg2>. The basic unit of IGT is the Illumination-Guided Attention Block (IGAB), which is composed of two layer normalization (LN), an IG-MSA, and a feed-forward network (FFN) <ref:2303. p. 206705#pg2>.
Where the gains come from
The key component of IGT is the Illumination-Guided Multi-head Self-Attention (IG-MSA), which exploits illumination representations to direct the computation of self-attention and enhance interactions between regions of different exposure levels <ref:2303.06705#pg2>. The complexity analysis shows that the computational complexity of IG-MSA is linear to the spatial size, O(IG-MSA) = 2HW C 2/k <ref:2303. p. 406705#pg2>. This contrasts with the quadratic complexity of global MSA used by some previous CNN-Transformer methods, O(G-MSA) = 2(HW) 2C <ref:2303. p. 406705#pg2>.
Ablation and performance validation
The ablation study on the SDSD-outdoor dataset shows that joint exploitation of ORF and IG-MSA achieves 3.37 dB improvements over baseline-1 <ref:2303. p. 706705#pg2>. Furthermore, the comparison with SOTA Transformer-based image restoration algorithms demonstrates that Retinexformer gains by 2. p. 573 to 4.26 dB on seven datasets <ref:2303. p. 506705#pg2>. Qualitative results confirm that Retinexformer can effectively remove noise without introducing spots and artifacts, while quantitative results show significant improvements on thirteen datasets <ref:2303. p. 506705#pg2>. The method also achieves the highest result on average in the user study scores <ref:2303. p. 606705#pg2>.
Low-light object detection
In low-light object detection experiments on the ExDark dataset, Retinexformer achieves an average precision (AP) score of 66.1, which is 0.5 AP higher than the recent best self-supervised method SCI [37] <ref:2303. p. 606705#pg2>. Retinexformer also yields the best results on five object categories: bicycle, boat, bottle, cat, and dog <ref:2303. p. 606705#pg2>. This demonstrates the practical values of the method in high-level vision understanding <ref:2303. p. 606705#pg2>.
Conclusion
In this paper, Retinexformer is proposed as a novel Transformer-based method for low-light image enhancement <ref:2303.06705#pg2>. The authors successfully introduce perturbation terms to the original Retinex model to formulate ORF, and then design an IGT utilizing illumination information to direct long-range dependencies <ref:2303. p. 706705#pg2>. Extensive experiments show that Retinexformer dramatically outperforms SOTA methods on thirteen datasets <ref:2303. p. 706705#pg2>. The results of user study and low-light detection also demonstrate the practical values of our method <ref:2303. p. 706705#pg2>.
Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.
Improvements for AI systems
-
Retinexformer can model long-range dependencies through an Illumination-Guided Transformer (IGT) that utilizes illumination representations to direct self-attention, which enhances interactions between regions of different lighting conditions.
-
The One-stage Retinex-based Framework (ORF) allows the system to estimate illumination information and then employ a corruption restorer to suppress noise, artifacts, under-/overexposure, and color distortion in a single training stage.
-
The Illumination-Guided Multi-head Self-Attention (IG-MSA) mechanism treats a single-channel feature map as a token to compute self-attention linearly with respect to the spatial size, achieving linear complexity O(IG-MSA) = 2HW C2/k, unlike global MSA's quadratic complexity.
-
The enhanced AI system can achieve significant improvements over state-of-the-art methods, such as 4.30 dB on seven benchmarks and over 6 dB on SID and SDSD datasets.
-
The Retinexformer demonstrates superior performance in low-light object detection, achieving the highest average precision (AP) score of 66.1 on the ExDark dataset, outperforming recent best self-supervised and fully-supervised methods.
Sources
Related papers
- Do Egocentric Video-Language Models Capture Both Hand- and Object-Centric Cues?
- PVRF: All-in-one Adverse Weather Removal via Prior-modulated and Velocity-constrained Rectified Flow
- DLWM: Diverse Latent World Models for Efficient Multimodal Reasoning
- Out of Sight, Still in Mind: Token Compression for Omni-LLMs
- From Formulas to Protocols: A Compositional View of Binary Target Segmentation Metrics
- Traffic Sign Recognition in Autonomous Driving: Dataset, Benchmark, and Field Experiment