FDIR: Fidelity-Anchored Decompressed Image Restoration for Human and Machine Perception
summary
The gist
Image restoration quality can be evaluated along three complementary facets: pixellevel fidelity, human perception, and machine preference The authors propose FDIR…
In short
FDIR is a two-stage image restoration architecture designed to balance three quality metrics: pixel fidelity, human perception, and machine preference. It uses a first stage to recover global structure and a second stage to refine details. Experiments show FDIR achieves the best fidelity and second-best perceptual scores under extreme compression while maintaining strong performance on downstream AI tasks.
Key concepts
- Tripartite Quality Assessment
- Image quality is evaluated using three distinct measures: pixel-level accuracy (fidelity), how natural it looks to humans (perception), and how well it preserves data for other AI systems (machine preference). These three aspects often conflict, making a single quality metric insufficient for a complete evaluation.
- Quality-Guided One-Step Flow Matching (QO-Flow)
- This is the first stage of FDIR. It uses Flow Matching to learn deterministic paths between different image distributions in latent space. This process efficiently recovers the overall semantic structure of the image in one forward pass, avoiding the randomness found in other diffusion models.
- Observation-Anchored Residual Estimation (FCDR)
- This is the second stage, which focuses on restoring fine details like textures. It works by predicting a small correction (residual) to an initial estimate of the image. This method ensures that restored high-frequency details are grounded in the input image's structure, preventing unrealistic or 'hallucinated' artifacts.
Terminology used across episodes
This episode discusses
- FDIR: Fidelity-Anchored Decompressed Image Restoration for Human and Machine Perception · Paper Radio
The paper
FDIR: Fidelity-Anchored Decompressed Image Restoration for Human and Machine Perception · Read on arXiv
Kuan-Yen Chen, Fang-Yi Su, Philip Chikontwe, Jung-Hsien Chiang
Department of Computer Science and Information Engineering, National Cheng Kung University · Department of Biomedical Informatics, Harvard Medical School
Transcript
Introduction to the show: ident: Electrical Engineering. The day in electrical engineering: signals, systems and the hardware that carries them.
Rosa: I'm Rosa.
Dev: And I'm Dev, here with Taro.
Rosa: We're looking at the paper: "FDIR: Fidelity-Anchored Decompressed Image Restoration for Human and Machine Perception".
Dev: Here's the gist of the paper. Image restoration quality can be evaluated along three complementary facets: pixellevel fidelity, human perception, and machine preference The authors propose FDIR, a two-stage architecture that decouples the conflicting demands through complementary inductive biases Extensive experiments demonstrate FDIR’s best fidelity and second-best perceptual scores under extreme compression, with leadership on 7/10 downstream tasks at practical compression ratios Tripartite Quality Assessment Image restoration quality is conventionally assessed along a single axis, typically fidelity or perceptual quality The authors argue that a complete evaluation requires three complementary axes
15, 16: , each capturing a distinct and irreducible aspect of restoration quality These axes are fidelity, human perception, and machine preference Fidelity measures pixel-level faithfulness to the ground truth through metrics such as PSNR and SSIM
25: Human perception captures whether restored images appear natural to human observers, decomposing into perceptual similarity and distributional realism Machine preference reflects how well restored images preserve semantic information for downstream vision systems
15: These three axes form a tension triangle, with fidelity-perception conflict being theoretically grounded
9: How the training signal is built FDIR decomposes the restoration trajectory into two stages with complementary roles (Figure 2) Stage 1, Quality-Guided One-Step Flow Matching (QO-Flow), recovers global semantic structure in latent space via a single forward pass This stage utilizes Flow Matching (FM)
19: to learn deterministic ODE trajectories between distributions, avoiding the stochastic drift inherent to SDE-based diffusion models The training objective for Stage 1 is the flow matching loss LFM = Et∼U
0,1: ,(zdeg,zclean,cQF)∼D∥vθ(zt, t, cQF) − (zclean − zdeg)∥2 Stage 2, Flow-Conditioned Detail Refinement (FCDR), deterministically restores high-frequency textures and suppresses generative hallucinations in pixel space FCDR is formulated as an observation-anchored residual estimation where xˆ = clip
−1,1: xdeg + ∆xθ(xdeg, xˆFM) The composite loss for Stage 2 is LFCDR = λrec Lrec + λperc Lperc + λstruct Lstruct, targeting fidelity, human perception, and edge structure preservation Where the gains come from The primary motivation for FDIR is a theoretically-grounded design for the tripartite tradeoff The authors connect the perception-distortion bound
9, 14: to machine preference
15, 16: , showing that a single estimator cannot jointly minimize all three Stage 2's observation-anchored residual formulation acts as an implicit regularizer by constraining the hypothesis space to a neighborhood of the degraded input, simultaneously suppressing hallucinations and oversmoothing The two stages contribute asymmetrically, with Stage 1’s restored semantic structure acting as a distributional regularizer that benefits downstream feature extractors Efficiency and Practicality FDIR achieves competitive performance with fewer training images and a single function evaluation (NFE = 1) Through LoRA-based adaptation of pre-trained flow matching priors, FDIR achieves competitive results using only ∼3.5K task-specific fine-tuning pairs, yielding faster inference than comparable one-step baselines Inference latency is benchmarked on RTX 4090 and FDIR achieves 47.4 ms, outperforming DiffBIR by 50× and CODiff (also one-step, same precision) by 2.5× Furthermore, the pipeline is practical because it conditions on the quality factor at inference time, which can be reliably recovered from JPEG headers with negligible latency Robustness and Generalization FDIR demonstrates robustness beyond JPEG by being retrained and evaluated on WebP using DIV2K-Val at QF=1, 5, 10 Table 6 reveals that FDIR maintains consistent gains (+1.59–1.63 dB) to show its robustness despite WebP’s distinct compression pipeline The method generalizes across foundation models, as both SD2.
Paper summary: Rosa: , so to wrap up, FDIR: Fidelity-Anchored Decompressed Image Restoration for Human and Machine Perception is proposing a two stage architecture that splits the restoration job between recovering global structure via flow matching and refining local details deterministically >
Dev: , it’s about balancing the tension between pixel level fidelity, human perception, and machine preference by using complementary inductive biases in each stage >
Taro: , what this means for us is that we can get an output that isn't just technically accurate but also looks natural to humans and performs well when other AI systems try to use it >
Rosa: , the authors of FDIR managed this balance through architectural innovation rather than just feeding it more data or scaling up the model size >
Dev: , they used transfer learning from pre trained flow matching priors, which allowed them to get competitive performance using only about three point five thousand task specific fine tuning pairs >
Taro: , that efficiency is something I can see a lot of value in when deploying these kinds of restoration systems in real world robotics where we have limited compute and data resources >
Rosa: , the main implication is that we don't have to choose between making an image look sharp and keeping it realistic, because this method aims for a balanced position >
Conclusion: Rosa: So we've looked at how this FDIR method splits image restoration into two stages to get good fidelity and also keep things looking natural for humans and machines >
Dev: Yeah, that split is what’s interesting because it tackles that tension between making a picture look technically perfect and making it look realistic >
Rosa: The title itself, Fidelity-Anchored Decompressed Image Restoration for Human and Machine Perception, shows they are targeting all three quality measures at once >
Dev: They're claiming this two-stage approach decouples those conflicting demands through complementary biases in the training process >
Rosa: It means instead of picking one way to optimize for sharpness or realism, they’ve built a system that handles both simultaneously >
Dev: And their experiments show that under really bad compression, FDIR actually achieves the best fidelity and second-best perceptual scores compared to other methods >
Rosa: That's what I mean when I say it can handle extreme compression better than the competition >
Dev: The authors also showed it performs well on downstream tasks for vision systems at practical compression ratios, which is important for real world deployment >
Rosa: It’s about showing that you don't have to sacrifice one aspect of quality to get another one >
Dev: What this implies is that we might see restoration tools that are more reliable across different kinds of visual analysis, not just pretty pictures >
Rosa: We've seen the architecture, but what does this actually mean for how we use these tools in practice?
More episodes
- Daily Summary for 2026-11-19
- 2111.07910-Mask-guided Spectral-wise Transformer for Efficient Hyperspectral Image Reconstruction
- 2203.02149-HDNet: High-resolution Dual-domain Learning for Spectral Compressive Imaging
- 2205.10102-Degradation-Aware Unfolding Half-Shuffle Transformer for Spectral Compressive Imaging
- 2311.10959-Structure-Aware Sparse-View X-ray 3D Reconstruction
- 2403.04116-Radiative Gaussian Splatting for Efficient X-ray Novel View Synthesis
- 2410.02572-Combining Pre- and Post-Demosaicking Noise Removal for RAW Video
- 2604.17453-Learned Nonlocal Feature Matching and Filtering for RAW Image Denoising
- 2603.20387-End-to-End Multi-Task Learning for Adjustable Joint Noise Reduction and Hearing Loss Compensation
- 2507.20825-Chirp-Permuted AFDM: A Versatile and Multifunctional Waveform for 6G