FDIR: Fidelity-Anchored Decompressed Image Restoration for Human and Machine Perception

arXiv:2608.00111 · eess.IV, cs.CV, cs.LG · Submitted 2026-07-31 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: Electrical Engineering. The day in electrical engineering: signals, systems and the hardware that carries them.

Rosa: I'm Rosa.

Dev: And I'm Dev, here with Taro.

Rosa: We're looking at the paper: "FDIR: Fidelity-Anchored Decompressed Image Restoration for Human and Machine Perception".

Dev: Here's the gist of the paper. Image restoration quality can be evaluated along three complementary facets: pixellevel fidelity, human perception, and machine preference The authors propose FDIR, a two-stage architecture that decouples the conflicting demands through complementary inductive biases Extensive experiments demonstrate FDIR’s best fidelity and second-best perceptual scores under extreme compression, with leadership on 7/10 downstream tasks at practical compression ratios Tripartite Quality Assessment Image restoration quality is conventionally assessed along a single axis, typically fidelity or perceptual quality The authors argue that a complete evaluation requires three complementary axes

15, 16: , each capturing a distinct and irreducible aspect of restoration quality These axes are fidelity, human perception, and machine preference Fidelity measures pixel-level faithfulness to the ground truth through metrics such as PSNR and SSIM

25: Human perception captures whether restored images appear natural to human observers, decomposing into perceptual similarity and distributional realism Machine preference reflects how well restored images preserve semantic information for downstream vision systems

15: These three axes form a tension triangle, with fidelity-perception conflict being theoretically grounded

9: How the training signal is built FDIR decomposes the restoration trajectory into two stages with complementary roles (Figure 2) Stage 1, Quality-Guided One-Step Flow Matching (QO-Flow), recovers global semantic structure in latent space via a single forward pass This stage utilizes Flow Matching (FM)

19: to learn deterministic ODE trajectories between distributions, avoiding the stochastic drift inherent to SDE-based diffusion models The training objective for Stage 1 is the flow matching loss LFM = Et∼U

0,1: ,(zdeg,zclean,cQF)∼D∥vθ(zt, t, cQF) − (zclean − zdeg)∥2 Stage 2, Flow-Conditioned Detail Refinement (FCDR), deterministically restores high-frequency textures and suppresses generative hallucinations in pixel space FCDR is formulated as an observation-anchored residual estimation where xˆ = clip

−1,1: xdeg + ∆xθ(xdeg, xˆFM) The composite loss for Stage 2 is LFCDR = λrec Lrec + λperc Lperc + λstruct Lstruct, targeting fidelity, human perception, and edge structure preservation Where the gains come from The primary motivation for FDIR is a theoretically-grounded design for the tripartite tradeoff The authors connect the perception-distortion bound

9, 14: to machine preference

15, 16: , showing that a single estimator cannot jointly minimize all three Stage 2's observation-anchored residual formulation acts as an implicit regularizer by constraining the hypothesis space to a neighborhood of the degraded input, simultaneously suppressing hallucinations and oversmoothing The two stages contribute asymmetrically, with Stage 1’s restored semantic structure acting as a distributional regularizer that benefits downstream feature extractors Efficiency and Practicality FDIR achieves competitive performance with fewer training images and a single function evaluation (NFE = 1) Through LoRA-based adaptation of pre-trained flow matching priors, FDIR achieves competitive results using only ∼3.5K task-specific fine-tuning pairs, yielding faster inference than comparable one-step baselines Inference latency is benchmarked on RTX 4090 and FDIR achieves 47.4 ms, outperforming DiffBIR by 50× and CODiff (also one-step, same precision) by 2.5× Furthermore, the pipeline is practical because it conditions on the quality factor at inference time, which can be reliably recovered from JPEG headers with negligible latency Robustness and Generalization FDIR demonstrates robustness beyond JPEG by being retrained and evaluated on WebP using DIV2K-Val at QF=1, 5, 10 Table 6 reveals that FDIR maintains consistent gains (+1.59–1.63 dB) to show its robustness despite WebP’s distinct compression pipeline The method generalizes across foundation models, as both SD2.

Paper summary: Rosa: , so to wrap up, FDIR: Fidelity-Anchored Decompressed Image Restoration for Human and Machine Perception is proposing a two stage architecture that splits the restoration job between recovering global structure via flow matching and refining local details deterministically >

Dev: , it’s about balancing the tension between pixel level fidelity, human perception, and machine preference by using complementary inductive biases in each stage >

Taro: , what this means for us is that we can get an output that isn't just technically accurate but also looks natural to humans and performs well when other AI systems try to use it >

Rosa: , the authors of FDIR managed this balance through architectural innovation rather than just feeding it more data or scaling up the model size >

Dev: , they used transfer learning from pre trained flow matching priors, which allowed them to get competitive performance using only about three point five thousand task specific fine tuning pairs >

Taro: , that efficiency is something I can see a lot of value in when deploying these kinds of restoration systems in real world robotics where we have limited compute and data resources >

Rosa: , the main implication is that we don't have to choose between making an image look sharp and keeping it realistic, because this method aims for a balanced position >

Conclusion: Rosa: So we've looked at how this FDIR method splits image restoration into two stages to get good fidelity and also keep things looking natural for humans and machines >

Dev: Yeah, that split is what’s interesting because it tackles that tension between making a picture look technically perfect and making it look realistic >

Rosa: The title itself, Fidelity-Anchored Decompressed Image Restoration for Human and Machine Perception, shows they are targeting all three quality measures at once >

Dev: They're claiming this two-stage approach decouples those conflicting demands through complementary biases in the training process >

Rosa: It means instead of picking one way to optimize for sharpness or realism, they’ve built a system that handles both simultaneously >

Dev: And their experiments show that under really bad compression, FDIR actually achieves the best fidelity and second-best perceptual scores compared to other methods >

Rosa: That's what I mean when I say it can handle extreme compression better than the competition >

Dev: The authors also showed it performs well on downstream tasks for vision systems at practical compression ratios, which is important for real world deployment >

Rosa: It’s about showing that you don't have to sacrifice one aspect of quality to get another one >

Dev: What this implies is that we might see restoration tools that are more reliable across different kinds of visual analysis, not just pretty pictures >

Rosa: We've seen the architecture, but what does this actually mean for how we use these tools in practice?

Kuan-Yen Chen, Fang-Yi Su, Philip Chikontwe, Jung-Hsien Chiang

Department of Computer Science and Information Engineering, National Cheng Kung University · Department of Biomedical Informatics, Harvard Medical School

eess.IV, cs.CV, cs.LG

Submitted: 2026-07-31

Updated: 2026-11-16

Comments: 16 pages, 8 figures, 14 tables. The first three authors contributed equally. v2: revised method and training, new experiments, and an appendix

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Importance score: 100/100

The gist: Image restoration quality can be evaluated along three complementary facets: pixellevel fidelity, human perception, and machine preference The authors propose FDIR…

Key concepts

Tripartite Quality Assessment
Image quality is evaluated using three distinct measures: pixel-level accuracy (fidelity), how natural it looks to humans (perception), and how well it preserves data for other AI systems (machine preference). These three aspects often conflict, making a single quality metric insufficient for a complete evaluation.
Quality-Guided One-Step Flow Matching (QO-Flow)
This is the first stage of FDIR. It uses Flow Matching to learn deterministic paths between different image distributions in latent space. This process efficiently recovers the overall semantic structure of the image in one forward pass, avoiding the randomness found in other diffusion models.
Observation-Anchored Residual Estimation (FCDR)
This is the second stage, which focuses on restoring fine details like textures. It works by predicting a small correction (residual) to an initial estimate of the image. This method ensures that restored high-frequency details are grounded in the input image's structure, preventing unrealistic or 'hallucinated' artifacts.

Terminology

Summary

Image restoration quality can be evaluated along three complementary facets: pixellevel fidelity, human perception, and machine preference The authors propose FDIR, a two-stage architecture that decouples the conflicting demands through complementary inductive biases Extensive experiments demonstrate FDIR’s best fidelity and second-best perceptual scores under extreme compression, with leadership on 7/10 downstream tasks at practical compression ratios

Tripartite Quality Assessment

Image restoration quality is conventionally assessed along a single axis, typically fidelity or perceptual quality The authors argue that a complete evaluation requires three complementary axes [15, 16], each capturing a distinct and irreducible aspect of restoration quality These axes are fidelity, human perception, and machine preference Fidelity measures pixel-level faithfulness to the ground truth through metrics such as PSNR and SSIM [25] Human perception captures whether restored images appear natural to human observers, decomposing into perceptual similarity and distributional realism Machine preference reflects how well restored images preserve semantic information for downstream vision systems [15] These three axes form a tension triangle, with fidelity-perception conflict being theoretically grounded [9]

How the training signal is built

FDIR decomposes the restoration trajectory into two stages with complementary roles (Figure 2) Stage 1, Quality-Guided One-Step Flow Matching (QO-Flow), recovers global semantic structure in latent space via a single forward pass This stage utilizes Flow Matching (FM) [19] to learn deterministic ODE trajectories between distributions, avoiding the stochastic drift inherent to SDE-based diffusion models The training objective for Stage 1 is the flow matching loss LFM = Et∼U[0,1],(zdeg,zclean,cQF)∼D∥vθ(zt, t, cQF) − (zclean − zdeg)∥2 Stage 2, Flow-Conditioned Detail Refinement (FCDR), deterministically restores high-frequency textures and suppresses generative hallucinations in pixel space FCDR is formulated as an observation-anchored residual estimation where xˆ = clip[−1,1]xdeg + ∆xθ(xdeg, xˆFM) The composite loss for Stage 2 is LFCDR = λrec Lrec + λperc Lperc + λstruct Lstruct, targeting fidelity, human perception, and edge structure preservation

Where the gains come from

The primary motivation for FDIR is a theoretically-grounded design for the tripartite tradeoff The authors connect the perception-distortion bound [9, 14] to machine preference [15, 16], showing that a single estimator cannot jointly minimize all three Stage 2's observation-anchored residual formulation acts as an implicit regularizer by constraining the hypothesis space to a neighborhood of the degraded input, simultaneously suppressing hallucinations and oversmoothing The two stages contribute asymmetrically, with Stage 1’s restored semantic structure acting as a distributional regularizer that benefits downstream feature extractors

Efficiency and Practicality

FDIR achieves competitive performance with fewer training images and a single function evaluation (NFE = 1) Through LoRA-based adaptation of pre-trained flow matching priors, FDIR achieves competitive results using only ∼3.5K task-specific fine-tuning pairs, yielding faster inference than comparable one-step baselines Inference latency is benchmarked on RTX 4090 and FDIR achieves 47. p. 14 ms, outperforming DiffBIR by 50× and CODiff (also one-step, same precision) by 2. p. 85× Furthermore, the pipeline is practical because it conditions on the quality factor at inference time, which can be reliably recovered from JPEG headers with negligible latency

Robustness and Generalization

FDIR demonstrates robustness beyond JPEG by being retrained and evaluated on WebP using DIV2K-Val at QF=1, 5, 10 Table 6 reveals that FDIR maintains consistent gains (+1.59–1.63 dB) to show its robustness despite WebP’s distinct compression pipeline The method generalizes across foundation models, as both SD2.

Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.

Improvements for AI systems

  1. Quality-Guided One-Step Flow Matching (Stage 1) recovers global semantic structure in latent space via a single forward pass, achieving faster inference than comparable one-step baselines while utilizing a linear probability path that biases velocity toward constancy for exact single Euler integration at inference.

  2. Flow-Conditioned Detail Refinement (Stage 2) deterministically restores high-frequency textures and suppresses generative hallucinations by formulating restoration as observation-anchored residual estimation, where the network predicts a correction anchored to the degraded input xdeg, preventing drift toward hallucinated modes.

  3. The FDIR architecture decouples conflicting demands by connecting the perception distortion bound to machine preference and showing that a single estimator cannot jointly minimize all three, enabling a latent-then-pixel decoupling that targets machine preference architecturally.

  4. The system can achieve leading performance on 7 out of 10 downstream tasks at practical compression ratios, demonstrating superiority over fidelity-oriented models by maintaining high fidelity while suppressing the over-smoothing and hallucination artifacts inherent in generative approaches.

  5. The FDIR pipeline is practically applicable to blind restoration scenarios because it conditions on the JPEG quality factor at inference time using off-the-shelf tools, making the assumption of known QF practical without requiring a dedicated trained model.

Related papers