HarnessIR: Harnessing Multimodal Foundation Models for Universal Real-World Image Restoration
cs.CV
Submitted: 2026-10-07
Updated: 2026-10-08
Code: https://github.com/PolyU-VCLab/HarnessIR
License: http://creativecommons.org/licenses/by/4.0/
The gist: Real-world low-quality images suffer from complex mixed degradations, including but not limited to noise, blur, atmospheric effects, etc.
Terminology
Abstract
Real-world low-quality images suffer from complex mixed degradations, including but not limited to noise, blur, atmospheric effects, etc. Recent agentic methods usually model real-world image restoration (Real-IR) as a sequential tool calling problem over task-specific single-degradation restoration models. This paradigm, however, is fundamentally limited because complex real-world degradations cannot be cleanly undone degradation by degradation, and the tool used for task-specific models caps the capability of the agent system. In this work, we present HarnessIR, an agentic framework for Real-IR by harnessing a multimodal foundation model (MFM) as the executor. HarnessIR consists of five stages: perception and diagnosis, on-demand tool invocation, prompt composition, execution, and verification-driven refinement. Unlike prior agentic Real-IR methods that rely on tool chains assembled from task-specific models, HarnessIR feeds the restoration requirements, the perceptual diagnosis, and the evidence into an MFM that performs restoration in a single pass, followed by verification stages to determine whether the result warrants further processing. Under our harness, off-the-shelf MFMs handle restoration tasks remarkably well, achieving state-of-the-art results on the widely used MiO100 synthetic benchmark. More importantly, by exploiting the strong generalization ability of MFMs, HarnessIR delivers compelling restoration quality on challenging real-world scenes where previous agentic IR systems often struggle. Codes is available at https://github.com/PolyU-VCLab/HarnessIR.
Sources
- HAT: Hybrid Attention Transformer for Image Restoration
- Vision Transformers for Single Image Dehazing
- Towards Effective Multiple-in-One Image Restoration: A Sequential and Prompt Learning Strategy
- Photo-Realistic Image Restoration in the Wild with Controlled Vision-Language Models
- GPT-4 Technical Report
- 4KAgent: Agentic Any Image to 4K Super-Resolution
- Qwen-Image Technical Report
- Can Nano Banana 2 Replace Traditional Image Restoration Models? An Evaluation of Its Performance on Image Restoration Tasks
- RealRestorer: Towards Generalizable Real-World Image Restoration with Large-Scale Image Editing Models
- PP-OCRv6: From 1.5M to 34.5M Parameters, Surpassing Billion-Scale VLMs on OCR Tasks
- Q-Agent: Quality-Driven Chain-of-Thought Image Restoration Agent through Robust Multimodal Large Language Model
- OPERA: An Agent for Image Restoration with End-to-End Joint Planning-Execution Optimization
- ReAct: Synergizing Reasoning and Acting in Language Models
- MemGPT: Towards LLMs as Operating Systems
- Self-Evolving Agentic Image Restoration via Deliberate Planning and Intuitive Execution
- Qwen3-VL Technical Report
- Toward Real-world Single Image Deraining: A New Benchmark and Beyond
- Real-world Noisy Image Denoising: A New Benchmark
- NTIRE 2025 the 2nd Restore Any Image Model (RAIM) in the Wild Challenge
Related papers
- Loss Knows Best: Detecting Annotation Errors in Videos via Loss Trajectories
- AnchorWeave: World-Consistent Video Generation with Retrieved Local Spatial Memories
- Benchmarking the Robustness of Foundation Models for Mammography under Domain Shift
- MambaX-Net: Dual-Input Mamba-Enhanced Cross-Attention Network for Longitudinal MRI Segmentation
- TeleOCR: Navigating Document Parsing Across Digital and Camera-Captured Documents
- A Survey on Efficient Vision-Language-Action Models