Beyond Uniform Restoration: Empowering All-in-One Restoration with Pixel-Level Multimodal Guidance

The paper Beyond Uniform Restoration: Empowering All-in-One Restoration with Pixel-Level Multimodal Guidance proposes MGN-AIR, a novel pixel-level restoration framework for all-in-one image restoration. The authors argue that existing all-in-one restoration methods typically apply a uniform restoration strategy across the entire image, neglecting the fact that different regions may suffer from distinct degradation types and varying degrees of severity. In contrast, their approach performs restoration at the pixel level, thereby enabling more fine-grained and precise control over the restoration process.

The method consists of three main components: a Visual Prompt Generation Module (VPGM), a Multimodal Guidance Module (MGM), and a Pixel-Level Restoration Module (PLRM). The VPGM learns to estimate a pixel-level visual prompt that reflects local degradation intensity, supervised by the pixel-wise difference between the degraded image and its ground truth. The MGM leverages both textual and visual prompts to provide global and local degradation cues, where the textual prompt (from frozen CLIP) conveys degradation type and the visual prompt indicates per-pixel severity. The PLRM then executes distinct restoration at each pixel guided by a restoration matrix, using standard 3×3 2D convolutions with dilated convolutions for heavily degraded pixels and spatial attention blocks for less-degraded pixels.

Experiments are conducted on multiple benchmarks. On the CDD11 composite degradation dataset, our approach MGN-AIR surpasses recent work MoCE-IR and OneRestore on average PSNR by 1.51 dB and 2.09 dB, respectively. On the five-degradation benchmark (denoising, dehazing, deraining, deblurring, low-light enhancement), the method achieving the best performance on average PSNR and SSIM, with 2.29 dB improvement in PSNR over baseline work PromptIR, and compared with most recent work DFPIR, our approach significantly improves the overall performance, especially on deblurring and deraining tasks, achieving 2.36 dB and 1.85 dB improvement in PSNR. On the three-degradation benchmark, The average PSNR of our approach also outperforms existing methods, achieving 34.61 dB average PSNR.

Ablation studies confirm that removing any component in MGN-AIR will greatly influence the performance, with the visual prompt being particularly crucial. Complexity comparisons show that "our proposed module consistently outperforms these variants, and our lightweight version Ours-s outperforms baselines while keeps similar parameter counts, confirming that the improvements are attributed to the specific design of the MGB. Visualizations demonstrate that the method greatly preserves the local details while removing degradations" in complex composite scenarios, recovering subtle structures that prior works miss.

Improvements to AI systems:

1. Adaptive, pixel-wise degradation-aware processing: Replace uniform, whole-image restoration pipelines with a system that dynamically allocates different computational operations per pixel based on estimated local degradation severity. The improved AI can use a lightweight predictor (like VPGM) to generate a per-pixel severity map, then route heavily degraded regions through high-capacity dilated convolutions and lightly degraded regions through efficient spatial attention, reducing wasted computation while boosting restoration accuracy on heterogeneous images.

2. Multimodal degradation cue fusion: Integrate both frozen text embeddings (for global degradation type, e.g., rain, haze) and per-pixel visual prompts (for local intensity) into a single restoration backbone. The improved system can accept a natural-language description of the degradation condition as an optional input, and combine it with pixel-level visual features to disambiguate mixed or overlapping degradations—enabling more precise control than text-only or image-only guidance.

3. Self-supervised visual prompt learning from reconstruction error: Train a prompt generator to predict per-pixel degradation intensity by directly minimizing the pixel-wise difference between the degraded input and the ground-truth clean image. The improved AI can learn to identify where and how much restoration is needed without manual annotation, and can generalize to unseen degradation mixtures by inferring severity from local contrast and texture statistics.

4. Composite degradation handling via pixel-level restoration matrix: Use a learned restoration matrix that modulates both the type and strength of operations at each spatial location. The improved system can simultaneously handle multiple degradation types (e.g., blur + noise + low light) in different image regions, applying deblurring in one area, denoising in another, and brightness enhancement in a third—all within a single forward pass, without task-specific switching.

5. Lightweight, high-accuracy variant for edge deployment: Adopt the Ours-s configuration that matches baseline parameter counts but retains the multimodal guidance and pixel-level routing. The improved AI can run on resource-constrained devices (e.g., mobile cameras, drones) while still outperforming heavier uniform models, enabling real-time, fine-grained restoration in practical applications.

6. Interpretable degradation localization: The visual prompt map (pixel-wise severity) can be output as a side product. The improved AI can provide a heatmap showing which regions are degraded and how severely, aiding in downstream tasks like autonomous driving (identifying foggy or rainy patches) or medical imaging (highlighting artifacts), and enabling human-in-the-loop verification of restoration decisions.

All-in-one image restoration is a unified low-level vision task that aims to effectively recover high-quality images from inputs degraded by various types and levels of corruption using a single model. Recent works have achieved remarkable progress by learning degradation-adaptive prompts or network architectures. However, these methods typically apply a uniform restoration strategy across the entire image, neglecting the fact that different regions may suffer from distinct degradation types and varying degrees of severity. In contrast, we propose to perform restoration at the pixel level, thereby enabling more fine-grained and precise control over the restoration process. Specifically, we present MGN-AIR, a novel pixel-level restoration framework for all-in-one image restoration. Our approach first learns to estimate a pixel-level visual prompt. Then, it leverages both textual and visual prompts to provide global and local degradation cues, guiding the model on where to look and how to restore at each pixel. We conduct extensive experiments on multiple all-in-one image restoration benchmarks, covering a wide range of tasks including denoising, deraining, deblurring, dehazing, desnowing, and low-light enhancement. Experimental results demonstrate that our proposed method consistently and significantly outperforms existing approaches.