Stochastic Interpolants via Conditional Dependent Coupling
cs.CV
Submitted: 2025-09-27
Updated: 2026-10-29
Comments: Accepted at NeurIPS 2026. 50 pages, including references, appendices, and checklist
License: http://creativecommons.org/licenses/by/4.0/
The gist: Existing image generation models face critical challenges regarding the trade-off between computation and fidelity.
Terminology
Abstract
Existing image generation models face critical challenges regarding the trade-off between computation and fidelity. Specifically, models relying on a pretrained Variational Autoencoder (VAE) suffer from information loss, limited detail, and the inability to support end-to-end training. In contrast, models operating directly in the pixel space incur prohibitive computational cost. Although cascade models can mitigate computational cost, stage-wise separation prevents effective end-to-end optimization, hampers knowledge sharing, and often results in inaccurate distribution learning within each stage. To address these challenges, we introduce a unified multi-stage generative framework formulated under the stochastic interpolant formalism with our Conditional Dependent Coupling strategy as a Flow-Matching variant. It decomposes the generative process into interpolant trajectories at multiple stages, ensuring accurate distribution learning while enabling end-to-end optimization. Importantly, the entire process is modeled as a single unified Diffusion Transformer, eliminating the need for disjoint modules and also enabling knowledge sharing. Under explicitly stated assumptions, we provably reduce both transport cost and asymptotic inference time; we empirically validate the underlying NFE-transport-cost and per-evaluation-cost relations. Extensive experiments demonstrate that our method achieves competitive fidelity and substantially lower wall-clock cost in the low-NFE regime across multiple resolutions.
Sources
- Building Normalizing Flows with Stochastic Interpolants
- Stochastic Interpolants: A Unifying Framework for Flows and Diffusions
- Stochastic interpolants with data-dependent couplings
- PixelFlow: Pixel-Space Generative Models with Flow
- An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
- Stochastic Flow Matching for Resolving Small-Scale Physics
- One Step Diffusion via Shortcut Models
- Mean Flows for One-step Generative Modeling
- Classifier-Free Diffusion Guidance
- Scalable Adaptive Computation for Iterative Generation
- FlexVAR: Flexible Visual Autoregressive Modeling without Residual Prediction
- Pyramidal Flow Matching for Efficient Video Generative Modeling
- Auto-Encoding Variational Bayes
- Fractal Generative Models
- Flow Matching for Generative Modeling
- Flow Matching Guide and Code
- CAD-VAE: Leveraging Correlation-Aware Latents for Comprehensive Fair Disentanglement
- GPT-4 Technical Report
- FlowAR: Scale-wise Autoregressive Image Generation Meets Flow Matching
- Autoregressive Model Beats Diffusion: Llama for Scalable Image Generation
Related papers
- TEA: Text Encoder Alignment for Robust Concept Erasure in Text-to-Image Models
- Do Egocentric Video-Language Models Capture Both Hand- and Object-Centric Cues?
- Aura: Consistent Multi-Subject Video Generation via VLM-Grounded Semantic Alignment
- PVRF: All-in-one Adverse Weather Removal via Prior-modulated and Velocity-constrained Rectified Flow
- DLWM: Diverse Latent World Models for Efficient Multimodal Reasoning
- Out of Sight, Still in Mind: Token Compression for Omni-LLMs