Consistency Regularised Gradient Flows for Inverse Problems
stat.ML, cs.CV, cs.LG
Submitted: 2026-05-08
Updated: 2026-11-09
Comments: Accepted to NeurIPS 2026
Code: https://github.com/LeviBorodenko/motionblur
License: http://creativecommons.org/licenses/by/4.0/
The gist: Vision-Language Latent Diffusion Models (LDMs) (Rombach et al., 2022) provide powerful generative priors for inverse problems.
Terminology
Abstract
Vision-Language Latent Diffusion Models (LDMs) (Rombach et al., 2022) provide powerful generative priors for inverse problems. However, existing LDM-based inverse solvers typically require a large number of neural function evaluations (NFEs) and backpropagation through large pretrained components, leading to substantial computational costs and, in some cases, degraded reconstruction quality. We propose a unified Euclidean-Wasserstein-2 gradient-flow framework that jointly performs posterior sampling and prompt optimization in the latent space through a single flow that aligns the prior and posterior with the observed data. Combined with few-step latent text-to-image models, this formulation enables low-NFE inference without backpropagation through autoencoders. Experiments across several canonical imaging inverse problems show that our method achieves state-of-the-art performance with significantly reduced computational cost.
Sources
- A Multiscale Perspective on Maximum Marginal Likelihood Estimation
- Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasets
- Mean Flows for One-step Generative Modeling
- Elucidating the Design Space of Diffusion-Based Generative Models
- Consistency Trajectory Models: Learning Probability Flow ODE Trajectory of Diffusion
- Understanding Diffusion Objectives as the ELBO with Simple Data Augmentation
- Auto-Encoding Variational Bayes
- Learning Representations for Automatic Colorization
- Knowledge Distillation in Iterative Generative Models for Improved Sampling Speed
- Latent Consistency Models: Synthesizing High-Resolution Images with Few-Step Inference
- Diff-Instruct: A Universal Approach for Transferring Knowledge From Pre-trained Diffusion Models
- Isambard-AI: a leadership class supercomputer optimised specifically for Artificial Intelligence
- Multistep Distillation of Diffusion Models via Moment Matching
- { Euclidean, Metric, and Wasserstein } Gradient Flows: an overview
- Deep Unsupervised Learning using Nonequilibrium Thermodynamics
- Denoising Diffusion Implicit Models
- Maximum Likelihood Training of Score-Based Diffusion Models
- LVTINO: LAtent Video consisTency INverse sOlver for High Definition Video Restoration
- Wan: Open and Advanced Large-Scale Video Generative Models
- Diffusion-GAN: Training GANs with Diffusion
Related papers
- Uniform-in-Time Weak Propagation-of-Chaos in Shallow Neural Networks
- Diffusion Models Observe Only Gradients: A Geometric Perspective on Score Matching Errors
- Quantifying Error Propagation and Model Collapse in Diffusion Models
- Behavior of prediction performance metrics with rare events
- Statistical Inference and Learning for Shapley Additive Explanations (SHAP)
- Control Variate Design for Score-Function Gradient Estimators in Importance-Weighted Variational Inference