QuantDemoire: Quantization with Outlier Aware for Image Demoir'eing

arXiv:2510.04066 · cs.CV · Submitted 2025-10-05 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: We're looking at the paper: "QuantDemoire: Quantization with Outlier Aware for Image Demoir'eing".

Jane: Here's the gist of the paper. Image demoireing is a low-level vision task aimed at removing colored stripes caused by frequency aliasing between display screens and imaging sensors The main contribution of this work is the proposal of QuantDemoire, a post-training quantization framework tailored for image demoireing that achieves large reductions in parameters and computation while maintaining quality Model quantization offers a compelling solution to the challenge of deploying deep learning-based demoireing models on edge devices due to its ability to compress weights and activations from 32-bit floating-point to low-bit integers Directly applying existing quantization methods introduces severe performance degradation in demoireing models because of distribution outliers and weakened representations in smooth regions Quantizer Design The framework introduces two key components to mitigate accuracy loss from two perspectives: the quantizer and the calibration 1.

Tom: Outlier-Aware Quantizer involves using random sampling to estimate and remove activation outliers, while keeping a very small fraction of extreme weights in FP16 precision with negligible overhead 2.

Jane: Mixed-Precision Weight Quantizer proposes preserving outliers in FP16 precision by calculating lower and upper percentile thresholds Tlow and Thigh to separate outlier weights from normal ones Calibration Strategy The frequency-aware calibration strategy addresses the issue of banding artifacts caused by low-bit quantization in smooth regions 1.

Tom: It explicitly emphasizes mid- and low-frequency components during the optimization of quantization parameters, which alleviates the banding in smooth regions caused by quantization 2.

Title and authors: Tom: So we’ve talked about what QuantDemoire is and what it does, focusing on its dual approach of outlier awareness and frequency calibration. Now let’s get into a slightly more detailed look at the summary they provide in the paper.

Jane: They summarize that their core idea is to mitigate accuracy loss from two perspectives: the quantizer and the calibration. This means they aren't just changing one setting, but fixing errors where they happen in two different places.

Lu: The summary clearly breaks down those two components: first, the outlier-aware quantizer handles activations by using random sampling to estimate and discard outliers while preserving a few extreme weights in FP16 with negligible overhead.

Meng: That sounds like a trade-off we discussed—saving computation but accepting some FP16 storage for those extreme weights. What’s the cost of that specific mechanism?

Jane: The cost is that they are keeping a very small fraction of those extreme weights in FP16 precision with negligible overhead, which they argue is manageable compared to the accuracy loss from discarding activation outliers.

Tom: And then there’s the second part, the calibration strategy, which explicitly emphasizes mid- and low-frequency components during the optimization of quantization parameters.

Lu: This frequency emphasis helps alleviate banding in smooth regions because they are optimizing those bounds specifically using components extracted through a frequency extraction process, which is defined by a kernel k.

Meng: So they are essentially telling the quantizer, "Hey, when you decide how to quantize these numbers, pay extra attention to what happens at the mid and low frequencies."

Jane: Exactly. It’s about making sure that the quantization choices don't destroy the visual consistency in those smooth areas where banding usually shows up.

Tom: So, to put it plainly for our listeners, QuantDemoire is a post-training quantization framework that intelligently manages where to keep precision—for both activations and weights—while using frequency information to ensure that smooth visual details aren't lost due to quantization errors.

Lu: The summary emphasizes that their method provides superior efficiency and performance compared to recent quantization methods on the UHDM dataset, with results showing a reduction in parameters and computation by over eighty-six point six percent at four-bit, with less than a four point seven percent performance drop in PSNR >

Meng: That comparison against the full precision model is what really makes it stand out when you think about edge deployment constraints.

Jane: It’s that massive reduction in size and computation that allows these models to run where they couldn't before, while still keeping the visual quality within acceptable limits.

Tom: So, we see this as a way to bridge the gap between high-quality vision models and the hardware limitations of edge devices through smarter quantization.

Lu: The paper outlines that their method consistently outperforms existing quantization methods quantitatively and visually across various datasets and bit-widths, which is what’s really important for researchers looking for solid evidence.

The paper's summary: Tom: Now let’s look at the specific improvements they propose in QuantDemoire beyond just the two main components we mentioned earlier. These are the technical tweaks that make it work together effectively.

Jane: They introduce three key technical enhancements to address those limitations, starting with that outlier-aware quantizer, which uses random sampling to estimate and remove activation outliers and keeps a few extreme weights in FP16 precision with negligible overhead.

Lu: That's the first improvement: handling the distribution outliers in activations by using sample-based estimation, which reduces errors compared to just blindly discarding them.

Meng: I’m still thinking about that FP16 storage for weights—how do they decide which ones get saved there? Is it based on some learned criteria or fixed thresholds?

Jane: They use a mixed-precision weight quantizer that proposes preserving outliers in FP16 precision by calculating lower and upper percentile thresholds, Tlow and Thigh to separate the outlier weights from the normal ones.

Tom: So they’re using statistical measures like percentiles to decide which weights deserve that extra precision storage, rather than just arbitrarily picking a few.

Lu: That threshold calculation is crucial because it allows them to systematically separate the outliers, which is a refinement over just keeping a fixed small fraction of extreme weights.

Meng: That sounds more robust for handling diverse data distributions, but what about the calibration side? How does that frequency-aware strategy improve things beyond just focusing on mid and low frequencies?

Jane: The improvement there is that they optimize the quantization parameters by explicitly emphasizing mid- and low-frequency components during the optimization of those parameters.

Tom: So they are fine-tuning the quantization boundaries specifically based on what matters most for visual quality in those frequency ranges, not just minimizing a general error function.

Lu: This is done through extracting mid- and low-frequency components from both the model outputs and the ground truth using a dedicated convolution kernel k to define an L step iterative procedure to optimize the quantization bounds.

Meng: The kernel k part sounds like it’s introducing another dependency, meaning we need to design or choose this specific extraction mechanism carefully for our actual models.

Jane: That’s right, the kernel k defines how they get those important frequency components, and the iterative procedure is how they use that information to set their quantization bounds.

Tom: So, the improvements boil down to having a principled way to handle outliers in activations with sampling, a statistically informed way to handle weight outliers with percentiles, and a frequency-aware calibration strategy that targets smooth region banding.

The paper's improvements: Jane: So we’ve gone through the details of QuantDemoire, which is this post-training quantization framework designed for image demoir'eing tasks. Overall, the paper shows a strong path toward achieving significant model compression while maintaining good quality.

Tom: It really does show that you can tackle the hard problems of distribution outliers and weak representations in smooth regions by having a dual approach: fixing activations with sampling and weights with mixed precision thresholds.

Lu: And then they layer on the frequency-aware calibration strategy to make sure those quantization choices don't ruin the visual fidelity in smooth areas, which is what’s really critical for image quality.

Meng: From my side, I’m seeing a promising path forward because the numbers show a huge efficiency gain, but I’ll be watching how well this holds up when you move from benchmark datasets to real-world production traffic.

Jane: And that's a fair point, Meng. The paper does acknowledge that their outlier-aware quantizer relies on random sampling for activation estimation, so it’s not a perfect measure of every single outlier.

Tom: Right, and they also have to manage the overhead of keeping those extreme weights in FP16 precision, so it’s not entirely free compression.

Lu: But the combination seems to balance that efficiency with accuracy effectively across different bit-widths, outperforming other methods on datasets like UHDM.

Meng: It’s a solid result if they can generalize this approach beyond the specific architecture of the model they tested, which is what we need for real-world deployment.

Jane: So, in summary, QuantDemoire offers a tailored solution for image demoir'eing by balancing efficiency through outlier awareness and calibration through frequency focus. It’s a framework to consider when you need to compress vision models without losing too much visual detail.

Conclusion: Tom: So we’ve looked at QuantDemoire, this framework for image demoir'eing that uses outlier awareness and frequency calibration to compress models while keeping quality high.

Jane: It seems like they’re really tackling the problem of deployment by making quantization smarter about where it applies precision.

Lu: They use an outlier-aware quantizer for activations, so they sample to estimate outliers and only keep a few extreme weights in higher precision with almost no extra cost.

Meng: And then on the weights, they use mixed precision by finding those lower and upper percentile thresholds to separate the outliers into FP16 storage. That sounds like a good way to manage that trade-off without just throwing everything into one bucket.

Lalam: I’m confident their method is solid because they didn't just try one thing; they addressed the distribution of activations and the distribution of weights separately with different statistical tools.

Tom: The calibration strategy is also key, too. They focus on mid- and low-frequency components during optimization to stop those banding artifacts in smooth regions.

Jane: That’s what I was curious about for image quality; making sure the quantization doesn't create ugly stripes when things are supposed to be smooth.

Lu: They achieve this by using a dedicated convolution kernel k to define an iterative procedure that optimizes the quantization boundaries specifically based on those frequencies.

Meng: From my point of view, if they can keep the parameter reduction over eighty-six percent while staying under four point seven percent performance drop, that’s a big win for running models on edge hardware.

Tom: It is a substantial reduction in computation and memory footprint for these vision models.

Jane: What does this mean for someone just using an AI application on their phone or laptop? It means those demoir'eing tasks can be done much faster and more efficiently without losing too much visual detail.

Lu: The implication is that we might see a wider range of high-quality vision models deployed on smaller devices because the quantization noise is being handled much more intelligently.

Lalam: I’m sure this approach to QuantDemoire will be referenced a lot because it solves two major headaches in post-training quantization simultaneously: activation outliers and frequency artifacts.

Tom: Yeah, it’s a very well-rounded solution for demoir'eing models right now.

Jane: It definitely shows that fine-tuning the quantization process with domain knowledge, like focusing on image frequencies, makes a real difference in visual output.

Lu: If we can see this level of control over the quantization boundaries applied to complex tasks like image demoir'eing, it opens up new avenues for optimizing other vision pipelines.

Meng: I’m just wondering how much computational overhead that iterative procedure adds during the actual inference run on a real mobile chip. That’s where I get nervous.

Tom: That’s a fair concern about the runtime cost of the calibration step itself.

Jane: Well, it sounds like they found a way to keep that trade-off manageable by focusing only on those important frequency bands during the optimization.

Lu: So, QuantDemoire is a solid piece of work for anyone looking to get serious compression gains for vision tasks that require high fidelity.

Lalam: That’s the paper we talked about: QuantDemoire: Quantization with Outlier Aware for Image Demoir'eing. Next time, we look at how Provably Safe, Yet Scalable Reinforcement Learning is handling its own stability issues on large-scale tasks.

Shanghai Jiao Tong University · Central Media Technology Institute, Huawei

cs.CV

Submitted: 2025-10-05

Updated: 2026-11-05

Comments: Accepted to NeurIPS 2026. Code is available at: https://github.com/zhengchen1999/QuantDemoire

Code: https://github.com/zhengchen1999/QuantDemoire

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Importance score: 97/100

The gist: Image demoireing is a low-level vision task aimed at removing colored stripes caused by frequency aliasing between display screens and imaging sensors The main contribution of this work is the…

Key concepts

Outlier-Aware Quantizer
This component estimates activation ranges using random sampling to identify and remove outliers. It keeps only a small fraction of extreme weights in FP16 precision, which helps reduce quantization errors without adding much computational overhead.
Mixed-Precision Weight Quantizer
Instead of quantizing all weights uniformly, this method preserves outlier weights by calculating lower (Tlow) and upper (Thigh) percentile thresholds. Weights falling outside these thresholds are stored in FP16 precision, balancing accuracy and efficiency.
Frequency-Aware Calibration Strategy
This strategy targets banding artifacts in smooth image regions caused by low-bit quantization. It optimizes quantization parameters by focusing optimization on mid- and low-frequency components extracted using a specific kernel (k) through an iterative process, ensuring better quality in these areas.

Terminology

Summary

Image demoireing is a low-level vision task aimed at removing colored stripes caused by frequency aliasing between display screens and imaging sensors The main contribution of this work is the proposal of QuantDemoire, a post-training quantization framework tailored for image demoireing that achieves large reductions in parameters and computation while maintaining quality

Motivation for Quantization

Model quantization offers a compelling solution to the challenge of deploying deep learning-based demoireing models on edge devices due to its ability to compress weights and activations from 32-bit floating-point to low-bit integers Directly applying existing quantization methods introduces severe performance degradation in demoireing models because of distribution outliers and weakened representations in smooth regions

Quantizer Design

The framework introduces two key components to mitigate accuracy loss from two perspectives: the quantizer and the calibration

  1. Outlier-Aware Quantizer involves using random sampling to estimate and remove activation outliers, while keeping a very small fraction of extreme weights in FP16 precision with negligible overhead

  2. Mixed-Precision Weight Quantizer proposes preserving outliers in FP16 precision by calculating lower and upper percentile thresholds Tlow and Thigh to separate outlier weights from normal ones

Calibration Strategy

The frequency-aware calibration strategy addresses the issue of banding artifacts caused by low-bit quantization in smooth regions

  1. It explicitly emphasizes mid- and low-frequency components during the optimization of quantization parameters, which alleviates the banding in smooth regions caused by quantization

  2. This is achieved by extracting mid- and low-frequency components from both the model outputs and the ground truth using a dedicated convolution kernel k to define an L-step iterative procedure

Experimental Validation

Extensive experiments validate that QuantDemoire achieves large reductions in parameters and computation while maintaining quality, consistently outperforming existing quantization methods across datasets and bit-widths

  • Compared with the full-precision model, our method reduces parameters and computation by over 86.6% at 4-bit with less than 4. p. 17% (PSNR) performance drop

  • QuantDemoire surpasses current SOTA quantization methods by more than 4 dB (PSNR, 4-bit) on the UHDM dataset

  • The frequency-aware calibration strategy significantly reduces banding artifacts caused by low-bit quantization

Conclusion

The proposed outlier-aware quantizer effectively addresses the limitations posed by extreme activation and weight values, and the frequency-aware calibration strategy mitigates uneven impact of quantization across frequency bands By employing random sampling for activation range estimation and selectively retaining a small fraction of weight outliers in higher precision, the method balances efficiency with accuracy The results show that QuantDemoire consistently outperforms existing quantization methods quantitatively and visually across various datasets and bit-widths

REFERENCES

arXiv:2510.04066v1 [cs. p. 0CV] 5 Oct 2025

--- Page 1 ---

Preprint.

--- Page 2 ---

Preprint.

--- Page 3 ---

Preprint.

--- Page 4 ---

Preprint.

--- Page 5 ---

Preprint.

--- Page 6 ---

Preprint.

--- Page 7 ---

Preprint.

--- Page 8 ---

Preprint.

--- Page 9 ---

Preprint.

--- Page 10 ---

Preprint.

--- Page 11 ---

Preprint.

Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.

Improvements for AI systems

  1. Outlier-Aware Quantizer for Activations: The system can reduce quantization errors by using random sampling to estimate activation ranges and discarding outliers while keeping a few extreme weights in FP16 with negligible cost.

  2. Mixed-Precision Weight Quantizer: The system can preserve the overall accuracy of quantized models by storing outlier weights separately in FP16 precision, specifically by calculating lower and upper percentile thresholds Tlow and Thigh for weight quantization (Equation 9).

  3. Frequency-Aware Calibration Strategy: The system can mitigate banding artifacts caused by low-bit quantization in smooth regions by optimizing quantization boundaries using mid- and low-frequency components extracted via an L step iterative procedure based on a kernel k (Equation 10).

Sources

Related papers