Combining Pre- and Post-Demosaicking Noise Removal for RAW Video
summary
The gist
Combining Pre- and Post-Demosaicking Noise Removal for RAW Video is a self-similaritybased denoising scheme that weights both a pre- and a postdemosaicking denoiser for Bayer-patterned CFA video data…
In short
The method combines two denoising steps for RAW video: one before and one after demosaicking Bayer-patterned data. This balance improves image quality, especially with high noise levels. It also uses temporal trajectory prefiltering to enhance texture recovery, making it competitive with current deep learning methods for real-world videography.
Key concepts
- Pre- and Post-Demosaicking Denoising
- This is a two-stage denoising process applied to RAW video data. The first stage denoises the data while it is still mosaicked (before demosaicking), and the second stage denoises the demosaicked image. Finding the right balance between these two stages significantly improves final image quality.
- Noise Model Estimation
- The paper estimates sensor noise using a normal approximation of a Poisson distribution. This requires capturing flat-field and bias frames to fit parameters (a and b) for different ISO settings. A local variance estimator is then computed during denoising to accurately model the noise in any region.
- Temporal Trajectory Prefiltering
- Before each denoising stage, the method uses motion trajectories from surrounding frames to prefilter the data. This involves calculating optical flow and spatially aligning frames. This step helps reconstruct fine details by exploiting temporal redundancy, particularly at high noise levels.
Terminology used across episodes
This episode discusses
The paper
Combining Pre- and Post-Demosaicking Noise Removal for RAW Video · Read on arXiv
Institute of Applied Computing and Community Code (IAC3) · University of les Illes Balears · Huawei Technologies France
Denoising is one of the fundamental steps of the processing pipeline that converts data captured by a camera sensor into a display-ready image or video. It is generally performed early in the pipeline, usually before demosaicking, although studies swapping their order or even conducting them jointly have been proposed. With the advent of deep learning, the quality of denoising algorithms has steadily increased. Even so, modern neural networks still have a hard time adapting to new noise levels and scenes, which is indispensable for real-world applications. With those in mind, we propose a self-similarity-based denoising scheme that weights both a pre- and a post-demosaicking denoiser for Bayer-patterned CFA video data. We show that a balance between the two leads to better image quality, and we empirically find that higher noise levels benefit from a higher influence pre-demosaicking. We also integrate temporal trajectory prefiltering steps before each denoiser, which further improve texture reconstruction. The proposed method only requires an estimation of the noise model at the sensor, accurately adapts to any noise level, and is competitive with the state of the art, making it suitable for real-world videography.
Transcript
Introduction to the show: ident: Electrical Engineering. The day in electrical engineering: signals, systems and the hardware that carries them.
Rosa: We're looking at the paper: "Combining Pre- and Post-Demosaicking Noise Removal for RAW Video".
Paper summary: Rosa: So we've just talked about the core thesis of this paper on combining pre and post demosaicking denoising for RAW video. Essentially, they show that mixing both types of denoisers works better than sticking to just one approach, because the optimal balance changes depending on how much noise is present in the scene.
Dev: And they found an empirical finding that higher noise levels actually benefit from a stronger influence of the pre-demosaicking stage. That means if you've got a lot of noise, you should lean into that initial filtering more heavily.
Taro: They also integrated these temporal trajectory prefiltering steps before each denoiser, which they claim further improves how texture is reconstructed in the final image.
Rosa: It sounds like the paper suggests that this combined self-similarity based scheme only needs an estimation of the noise model at the sensor and can adapt to any noise level, making it competitive with what's out there now.
Dev: And they compared their method against state-of-the-art deep learning methods, finding it outperforms FlexISP, RViDeNet and UDVD in all metrics.
Taro: They also matched or surpassed MaskDnGAN in PSNR for ISOs between three thousand two hundred and twelve thousand eight hundred and SSIM and MS-SSIM for the range of one thousand six hundred to six thousand four hundred while staying close to it in VMAF.
Rosa: So what they're really highlighting is that this approach offers a good trade-off when you need robust denoising across different kinds of sequences and noise levels that current neural networks still struggle with.
Dev: It really puts the focus on making classical video denoising algorithms viable alternatives for general video tasks, especially considering the computational constraints of mobile devices mentioned in their work.
Conclusion: Rosa: To wrap up, this paper about combining pre and post demosaicking denoising for RAW Video by Sánchez-Beeckman et al. shows that a balanced approach is superior to using just one stage of denoising before or after demosaicking.
Dev: The key finding they highlight is that the optimal balance depends on the noise level, specifically suggesting that you should increase the RAW filtering before demosaicking when you're dealing with higher ISO values.
Taro: They also emphasized how integrating those temporal trajectory prefiltering steps helps a lot with recovering fine details, especially when motion models are well-restricted to keep things aligned.
Rosa: And what this means for us is that this method only requires estimating the noise model at the sensor, which can be done with a simple camera calibration procedure, allowing accurate noise removal regardless of how high the noise level gets.
Dev: It's a practical point because it makes it suitable for real-world videography, meaning you don't need massive datasets to train something that works across all conditions.
Taro: The paper also flags a limitation: they found that with strong restrictions on the motion model, this temporal prefiltering step is key for detail recovery, but if those motion models aren't tight enough, the misalignment could cause issues.
Rosa: That's the trade-off they identified; you get better detail recovery by adding that temporal step, but you have to be careful about how strictly you define the motion model.
Dev: So in a nutshell, this work is about giving us a robust way to clean RAW video data by intelligently combining different denoising techniques while keeping the complexity manageable for practical use.
More episodes
- Daily Summary for 2026-11-19
- 2111.07910-Mask-guided Spectral-wise Transformer for Efficient Hyperspectral Image Reconstruction
- 2203.02149-HDNet: High-resolution Dual-domain Learning for Spectral Compressive Imaging
- 2205.10102-Degradation-Aware Unfolding Half-Shuffle Transformer for Spectral Compressive Imaging
- 2311.10959-Structure-Aware Sparse-View X-ray 3D Reconstruction
- 2403.04116-Radiative Gaussian Splatting for Efficient X-ray Novel View Synthesis
- 2604.17453-Learned Nonlocal Feature Matching and Filtering for RAW Image Denoising
- 2603.20387-End-to-End Multi-Task Learning for Adjustable Joint Noise Reduction and Hearing Loss Compensation
- 2507.20825-Chirp-Permuted AFDM: A Versatile and Multifunctional Waveform for 6G
- 2509.17516-Audiobook-CC: Controllable Long-context Speech Generation for Multicast Audiobook