Distribution Matching Distillation without Fake Score Network
Listen
Radio episode about this paper
Transcript
Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.
Tom: I'm Tom.
Jane: And I'm Jane, here with Lu, Meng and our in-house large language model, Lalam.
Tom: We're looking at the paper: "Distribution Matching Distillation without Fake Score Network".
Jane: Here's the gist of the paper. Distribution Matching Distillation (DMD) provides an effective distribution-level correction for few-step generation, while relying on an auxiliary fake-score network to track the evolving generative distribution.
Tom: In this work, we study whether this explicit tracker can be avoided when the generator itself has a flow-map structure.
Jane: We propose Fake-Score-network-Free DMD (FSF-DMD), a DMD formulation for flow-map generators that replaces the auxiliary fake-score estimator with a generator-induced pseudo-velocity surrogate.
Tom: The key observation is that the endpoint pseudo-velocity of a flow-map generator provides a tractable proxy for fakevelocity estimation, allowing the generator itself to supply the reverse-divergence signal.
Paper summary: Jane: We've looked at how the authors approach this problem with Distribution Matching Distillation without Fake Score Network and they claim they can avoid the need for an auxiliary fake score network when using flow map generators.
Tom: So what is their main thesis here, Jane? What are they trying to prove with this work?
Jane: The thesis is that the explicit tracker used in standard DMD methods adds unnecessary memory and update overhead, and they show that you can replace that auxiliary fake score estimator with a generator induced pseudo velocity surrogate when the generator has a flow map structure.
Lu: They are studying whether this explicit tracker can be avoided when the generator itself has a flow map structure, which is what makes their proposal specific to certain types of generators.
Meng: So if I'm working with a standard diffusion model that isn't structured as a flow map, this doesn't apply at all to me?
Lalam: Not necessarily, Meng; the paper specifically focuses on flow-map generators and proposes FSF-DMD as the solution for that structure.
Tom: It matters because they are combining forward divergence training with reverse divergence correction in a way that avoids the separate fake score training loop.
Conclusion: Tom: So to wrap up, we've been talking about Distribution Matching Distillation without Fake Score Network and how FSF-DMD works for flow map generators.
Jane: The main implication is that if you are using a flow map generator, you can leverage the generator's internal structure to provide the necessary fake score correction without needing a separate network dedicated to tracking distribution changes.
Lu: It keeps the DMD view that distillation is driven by a teacher-fake distributional correction, but instead of an auxiliary fake score estimator, they use this generator induced surrogate.
Meng: From a deployment standpoint, it means less moving parts in the model architecture because you aren't adding another separate network or training loop just for velocity estimation.
Lalam: The resulting objective extends to flow map consistent backward simulation and to a self teacher variant for training from scratch, showing how robust this formulation is.
Tom: So, it suggests that flow map generators can supply a useful fake score correction in DMD while preserving the distribution matching perspective by replacing the auxiliary estimator with a generator induced surrogate.
Seoul National University · Sogang University
cs.CV
Submitted: 2026-05-19
Updated: 2026-11-05
Comments: Accepted to NeurIPS 2026. 22 pages, 5 figures, 14 tables. Project page: https://snu-vgilab.github.io/FSF-DMD
Code: https://github.com/hustvl/LightningDiT
License: http://creativecommons.org/licenses/by-nc-sa/4.0/
Importance score: 99/100
The gist: Distribution Matching Distillation (DMD) provides an effective distribution-level correction for few-step generation…
Key concepts
- Distribution Matching Distillation (DMD)
- A technique used to correct the distribution of few-step generation by using a teacher model's fake score. It aims to align the generated distribution with a target distribution by leveraging this distributional correction during training.
- Flow-Map Structure
- A specific structure inherent in some generative models where the generation process can be described as a flow map. This structure allows the generator itself to provide information about how to reverse the transformation, which is key to replacing external score estimators.
- Pseudo-velocity Surrogate
- The endpoint pseudo-velocity of a flow-map generator is used as a substitute for the auxiliary fake velocity estimator. Because of mathematical properties related to PF-ODE coupling, this endpoint velocity acts as a tractable proxy, enabling the generator to supply the required reverse divergence signal without needing an extra network.
Terminology
Summary
Distribution Matching Distillation (DMD) provides an effective distribution-level correction for few-step generation, while relying on an auxiliary fake-score network to track the evolving generative distribution. p. 0 kr, jaesik. kr
Abstract
Distribution Matching Distillation (DMD) provides an effective distribution-level
correction for few-step generation, while relying on an auxiliary fake-score network
to track the evolving generative distribution. The fake-score estimator remains an additional
component with memory and update overhead. In this work, we study whether this
explicit tracker can be avoided when the generator itself has a flow-map structure. In this work, we study whether this
explicit tracker can be avoided when the generator itself has a flow-map structure. We propose Fake-Score-network-Free DMD (FSF-DMD), a DMD formulation
for flow-map generators that replaces the auxiliary fake-score estimator with a
generator-induced pseudo-velocity surrogate. We propose Fake-Score-network-Free DMD (FSF-DMD), a DMD formulation
for flow-map generators that replaces the auxiliary fake-score estimator with a
generator-induced pseudo-velocity surrogate. The key observation is that the endpoint pseudo-velocity of a flow-map generator provides a tractable proxy for fakevelocity estimation, allowing the generator itself to supply the reverse-divergence
signal.Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Methodology and Formulation
The authors formulate FSF-DMD by replacing the auxiliary fake-score estimator with a generator-induced surrogate, which is derived from the flow-map structure. This replacement involves using the endpoint pseudo-velocity of a flow-map generator as a surrogate for fakevelocity estimation. p. 0 We propose Fake-Score-network-Free DMD (FSF-DMD), a DMD formulation
for flow-map generators that replaces the auxiliary fake-score estimator with a
generator-induced pseudo-velocity surrogate. The key observation is that the endpoint pseudo-velocity of a flow-map generator provides a tractable proxy for fakevelocity estimation, allowing the generator itself to supply the reverse-divergence
signal. FSF-DMD keeps the
DMD perspective which is driven by a teacher-fake distributional correction, while replacing the
explicit fake-score network with a generator-induced surrogate. We also extend this formulation to
backward simulation for flow maps, and to a self-teacher variant for training from scratch. The final
objective combines this term with distribution matching in Eq. (14), weighted by a scalar λ:
Lbase = LCD + λLFSF-DMD(F, Φ), (15)
LCD = Ex,z,t,s "
wt,s
Fθ(xt;t, s) − sg
vΦ(xt;t) + (s − t)
d
dtFθ(xt;t, s)
2
2
#
.Where the paper says it
Where the paper says it
Where the paper says it
Derivation of the Surrogate Signal
The derivation shows that under the PF-ODE-based coupling, the fake velocity reduces to the endpoint pseudo-velocity Fθ(ˆxt;t, 0) when using a MeanFlow-style generator formulation. This occurs because the posterior distribution collapses to a point mass p(ˆxxˆt) = δ(f(ˆx; 0, t) − xˆt), leading to the identity vfake(ˆxt;t) = Fθ(ˆxt;t, 0). p. 4 By injectivity
and invertibility of the flow map, the posterior collapses to a point mass p(ˆx|xˆt) = δ(f(ˆx; 0, t) − xˆt),
yielding the identity for the fake velocity vfake (see Sec. 3):
vfake(ˆxt;t) = Exˆ|xˆt
[t
−1
(ˆxt − xˆ)] = t
−1
(ˆxt − f(ˆxt;t, 0)) (10)
≈ t
−1
(ˆxt − fθ(ˆxt;t, 0)) (11)
= Fθ(ˆxt;t, 0) = (ˆxt − fθ(ˆxt;t, 0))/t (12)
Therefore, under the PF-ODE coupling (ˆx, f(ˆx; 0, t)), vfake(ˆxt;t) is approximated by the endpoint
pseudo-velocity Fθ(ˆxt;t, 0) without introducing a separate network or training loop. By injectivity
and invertibility of the flow map, the posterior collapses to a point mass p(ˆx|xˆt) = δ(f(ˆx; 0, t) − xˆt),
yielding the identity for the fake velocity vfake (see Sec. 3):
vfake(ˆxt;t) = Exˆ|xˆt
[t
−1
(ˆxt − xˆ)] = t
−1
(ˆxt − f(ˆxt;t, 0)) (10)
≈ t
−1
(ˆxt − fθ(ˆxt;t, 0)) (11)
= Fθ(ˆxt;t, 0) = (ˆxt − fθ(ˆxt;t, 0))/t (12)
Therefore, under the PF-ODE coupling (ˆx, f(ˆx; 0, t)), vfake(ˆxt;t) is approximated by the endpoint
pseudo-velocity Fθ(ˆxt;t, 0) without introducing a separate network or training loop.Where the paper says it
Where the paper says it
Experimental Results and Comparisons
Experiments on ImageNet-1K 256×256 show that FSF-DMD improves flow-map baselines and reaches a lower FID than the listed DMD2 comparisons in the flow-map-initialized setting. Specifically, FSF-DMD improves the consistency distillation baseline from 6. p. 1 Our experiments
are intended as a proof of concept for the proposed network-free replacement, providing evidence
for its practical effectiveness across the studied initialization conditions. In the flow-map-initialized
comparison, FSF-DMD improves the consistency distillation baseline and reaches a lower FID than
the listed DMD2 comparisons, without an explicit fake-score network. FSF-DMD improves the consistency distillation
baseline from 6. Under this controlled comparison, FSF-DMD obtains a lower
FID than the listed variants without an explicit
fake-score network or an adversarial branch. In the flow-map-initialized
comparison, FSF-DMD improves the consistency distillation baseline and reaches a lower FID than
the listed DMD2 comparisons, without an explicit fake-score network. We further evaluate FSF-DMD
under flow-matching initialization and even in a training-from-scratch setting, where the tracker-free
correction remains effective in the studied settings.Where the paper says it
Where the paper says it
Where the paper says it
Practical Implications and Efficiency Gains
The replacement of the explicit fake velocity estimator with an endpoint pseudo-velocity provides several practical implications. Since the fake-side surrogate is computed from the current generator, it changes with the evolving generator without requiring a separately trained tracker. p. 1 This replacement has several practical implications. (i) Since the fake-side surrogate is computed from
the current generator, it changes with the evolving generator without requiring a separately trained
tracker. It therefore avoids adding a separate fakevelocity objective or branch to the generator, keeping its capacity focused on the generation task. 3 Implementation
The implemented objective keeps the usual independent perturbation xˆt = (1 − t)ˆx + tz′
for teacher
evaluation and inserts the generator-induced fake-side surrogate into a stop-gradient objective. Practically, the VRAM requirement was 60GB for DMD2 and 49GB for FSF-DMD when M = 1. The wall-clock speed was 1. The wall-clock speed was 1. 32 sec/step for DMD2 and 0.Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Conclusion
FSF-DMD suggests that flow-map generators can supply a useful fake-score correction in DMD while preserving the distribution-matching perspective. The method keeps the DMD view that distillation is driven by a teacher-fake distributional correction, but replaces the auxiliary fake-score estimator with a generator-induced surrogate. p. 8 5 Conclusion
We presented FSF-DMD, a fake-score-network-free DMD formulation for flow-map generators. The
method keeps the DMD view that few-step generators benefit from a teacher-fake distributional
correction, but replaces the auxiliary fake-score estimator with a generator-induced surrogate. This
frames the flow-map structure not only as a forward-divergence training mechanism, but also as a
way to provide the reverse-divergence signal without an explicit fake-score network. The resulting
objective extends to flow-map-consistent backward simulation and to a self-teacher variant for training
from scratch. This
frames the flow-map structure not only as a forward-divergence training mechanism, but also as a
way to provide the reverse-divergence signal without an explicit fake-score network. The resulting
objective extends to flow-map-consistent backward simulation and to a self-teacher variant for training
from scratch. Figure 4: 2-step FSF-DMD samples on
ImageNet-1K 256, flow-map-initialized. In summary, the experiments support FSF-DMD as
a distribution-matching method without an explicit
fake-score network for flow-map generators.Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
References
[1] Nicholas Matthew Boffi, Michael Samuel Albergo, and Eric Vanden-Eijnden. Flow map matching with stochastic interpolants: A mathematical framework for consistency models. p. 9 References
[1] Nicholas Matthew Boffi, Michael Samuel Albergo, and Eric Vanden-Eijnden. Flow map
matching with stochastic interpolants: A mathematical framework for consistency models.Where the paper says it
[2] Zhenglin Cheng, Peng Sun, Jianguo Li, and Tao Lin. Twinflow: Realizing one-step generation on large models with self-adversarial flows. p. 9 [2] Zhenglin Cheng, Peng Sun, Jianguo Li, and Tao Lin. Twinflow: Realizing one-step generation
on large models with self-adversarial flows.Where the paper says it
[3] Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A largescale hierarchical image database. p. 9 [3] Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A largescale hierarchical image database. Imagenet: A largescale hierarchical image database. In 2009 IEEE conference on computer vision and pattern
recognition, pages 248–255.Where the paper says it
Where the paper says it
[4] Kevin Frans, Danijar Hafner, Sergey Levine, and Pieter Abbeel. One step diffusion via shortcut models. p. 9 [4] Kevin Frans, Danijar Hafner, Sergey Levine, and Pieter Abbeel. One step diffusion via shortcut
models.Where the paper says it
[5] Xingtong Ge, Xin Zhang, Tongda Xu, Yi Zhang, Xinjie Zhang, Yan Wang, and Jun Zhang. Senseflow: Scaling distribution matching for flow-based text-to-image distillation. p. 9 [5] Xingtong Ge, Xin Zhang, Tongda Xu, Yi Zhang, Xinjie Zhang, Yan Wang, and Jun Zhang. Senseflow: Scaling distribution matching for flow-based text-to-image distillation.Where the paper says it
[6] Zhengyang Geng, Mingyang Deng, Xingjian Bai, J Zico Kolter, and Kaiming He. p. 9 In The
Fourteenth International Conference on Learning Representations, 2026. [6] Zhengyang Geng, Mingyang Deng, Xingjian Bai, J Zico Kolter, and Kaiming He. [6] Zhengyang Geng, Mingyang Deng, Xingjian Bai, J Zico Kolter, and Kaiming He. Mean
flows for one-step generative modeling.Where the paper says it
Where the paper says it
[7] Zhengyang Geng, Yiyang Lu, Zongze Wu, Eli Shechtman, J. Zico Kolter, and Kaiming He. Improved mean flows: On the challenges of fastforward generative models. p. 9 Zico Kolter, and Kaiming He. Improved mean flows: On the challenges of fastforward generative models.Where the paper says it
[8] Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by a two time-scale update rule converge to a local nash equilibrium. p. 9 [8] Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by a two time-scale update rule converge to a local nash equilibrium. Gans trained by a two time-scale update rule converge to a local nash equilibrium. Von Luxburg, S. Von Luxburg, S. Wallach, R. Wallach, R. Vishwanathan, and R.Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
[9] Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. p. 9 [9] Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Denoising diffusion probabilistic models. Larochelle, M. Larochelle, M. Ranzato, R. Ranzato, R. Hadsell, M. Hadsell, M. Balcan, and H.Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
[10] Jonathan Ho and Tim Salimans. Classifier-free diffusion guidance. p. 9 [10] Jonathan Ho and Tim Salimans. Classifier-free diffusion guidance.Where the paper says it
[11] Xun Huang, Zhengqi Li, Guande He, Mingyuan Zhou, and Eli Shechtman. Self forcing: Bridging the train-test gap in autoregressive video diffusion. p. 9 [11] Xun Huang, Zhengqi Li, Guande He, Mingyuan Zhou, and Eli Shechtman. Self forcing:
Bridging the train-test gap in autoregressive video diffusion.Where the paper says it
[12] Dongjun Kim, Chieh-Hsin Lai, Wei-Hsiang Liao, Naoki Murata, Yuhta Takida, Toshimitsu Uesaka, Yutong He, Yuki Mitsufuji, and Stefano Ermon. Consistency trajectory models: Learning probability flow ODE trajectory of diffusion. p. 9 [12] Dongjun Kim, Chieh-Hsin Lai, Wei-Hsiang Liao, Naoki Murata, Yuhta Takida, Toshimitsu
Uesaka, Yutong He, Yuki Mitsufuji, and Stefano Ermon. Consistency trajectory models:
Learning probability flow ODE trajectory of diffusion.Where the paper says it
[13] Tuomas Kynkäänniemi, Tero Karras, Samuli Laine, Jaakko Lehtinen, and Timo Aila. Improved precision and recall metric for assessing generative models. p. 9 [13] Tuomas Kynkäänniemi, Tero Karras, Samuli Laine, Jaakko Lehtinen, and Timo Aila. Improved
precision and recall metric for assessing generative models. Von Luxburg, S. Wallach, R. Denoising diffusion probabilistic models. Larochelle, M. Larochelle,
A. Beygelzimer, F. d'Alché-Buc, E. Fox, and R.Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
[14] Kyungmin Lee, Sihyun Yu, and Jinwoo Shin. Decoupled meanflow: Turning flow models into flow maps for accelerated sampling. p. 9 [14] Kyungmin Lee, Sihyun Yu, and Jinwoo Shin. Decoupled meanflow: Turning flow models into
flow maps for accelerated sampling.Where the paper says it
[15] Zichong Li, Liming Liu, Chen Liang, Weizhu Chen, and Tuo Zhao. Normuon: Making muon more efficient and scalable, 2025
[16] Yaron Lipman, Ricky T. Q. Chen, Heli Ben-Hamu, Maximilian Nickel, and Matthew Le. p. 9 [16] Yaron Lipman, Ricky T. Chen, Heli Ben-Hamu, Maximilian Nickel, and Matthew Le. Chen, Heli Ben-Hamu, Maximilian Nickel, and Matthew Le. Flow
matching for generative modeling.Where the paper says it
Where the paper says it
[17] Xingchao Liu, Chengyue Gong, and qiang liu. Flow straight and fast: Learning to generate and transfer data with rectified flow. p. 2 Most flow map models are governed by forward
divergence objectives, which are defined on real data and are known to encourage mode-covering. These objectives often spread out densities and may generate samples from low-likelihood regions.Where the paper says it
[18] Cheng Lu and Yang Song. Simplifying, stabilizing and scaling continuous-time consistency models. p. 3 (3)
Recent work [1] views this as a unifying object that consistency models [4, 6, 24] aim to learn. This
flow map has useful properties, injectivity and invertibility, when v
∗
t
(xt) is Lipschitz continuous.Where the paper says it
[19] Yanzuo Lu, Yuxi Ren, Xin Xia, Shanchuan Lin, Xing Wang, Xuefeng Xiao, Andy J. Ma, Xiaohua Xie, and Jian-Huang Lai. p. 9 In NeurIPS 2021 Workshop
on Deep Generative Models and Downstream Applications, 2021. [11] Xun Huang, Zhengqi Li, Guande He, Mingyuan Zhou, and Eli Shechtman. Diffusion models [9, 25] and flow matching models [16, 17] are
generative models that iteratively transform a prior distribution into a data distribution. They have
achieved remarkable success in high-fidelity image synthesis, while their multi-step sampling process
limits the generation speed.Where the paper says it
Where the paper says it
[20] Amirmojtaba Sabour, Sanja Fidler, and Karsten Kreis. Align your flow: Scaling continuoustime flow map distillation. p. 5 Our Fake-Score-network-Free DMD (FSF-DMD) objective becomes:
LFSF-DMD(F, Φ) = Ez,t h
wt ∥Fθ(z; 1, 0) − sg[Fθ(z; 1, 0) − ∆F,Φ(ˆxt, t)]∥
2
2
i
, (14)
where wt is a scalar weight. Since the surrogate is derived under the flow-map generator, we use
consistency distillation [20], LCD, to align the generator with the flow-map structure.Where the paper says it
[21] Tim Salimans, Ian Goodfellow, Wojciech Zaremba, Vicki Cheung, Alec Radford, Xi Chen, and Xi Chen. Improved techniques for training gans. p. 8 First, the generator is already capable of few-step generation, and the
correction signal may be more reliable since the generated samples remain closer to the teacher
distribution even at the early training stage. Second, the improved MeanFlow (iMF) model uses more
parameters and computation, and part of the gap may simply reflect scaling. Gans trained by a two time-scale update rule converge to a local nash equilibrium. Von Luxburg, S.Where the paper says it
Where the paper says it
[22] Tim Salimans, Thomas Mensink, Jonathan Heek, and Emiel Hoogeboom. Multistep distillation of diffusion models via moment matching. p. 2 This refines generated samples without an explicit fake-side network, while an expressivity
overhead in the generator network may arise. Moment Matching Distillation [22] approximates
the distributional difference via a first-order expansion in parameter space.Where the paper says it
[23] Neta Shaul, Uriel Singer, Itai Gat, and Yaron Lipman. Transition matching: Scalable and flexible generative modeling. p. 9 Chen, Heli Ben-Hamu, Maximilian Nickel, and Matthew Le. Flow
matching for generative modeling.Where the paper says it
[24] Yang Song, Prafulla Dhariwal, Mark Chen, and Ilya Sutskever. Consistency models. p. 0 1 Introduction
Diffusion models [9, 25] and flow matching models [16, 17] have achieved remarkable progress in
high-fidelity generation, while relying on multi-step sampling. To reduce generation cost, prior work
has focused on few-step generation through consistency-based flow-map models [1, 6, 12, 24] and
distribution matching distillation (DMD) [29, 30].Where the paper says it
[25] Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. p. 9 [6] Zhengyang Geng, Mingyang Deng, Xingjian Bai, J Zico Kolter, and Kaiming He. Mean
flows for one-step generative modeling.Where the paper says it
[26] Peng Sun and Tao Lin. Any-step generation via n-th order recursive consistent velocity field estimation. p. 0 The key observation is that the endpoint pseudo-velocity of a flow-map generator provides a tractable proxy for fakevelocity estimation, allowing the generator itself to supply the reverse-divergence
signal. Building on this observation, we derive a practical objective, extend it with
flow-map-consistent backward simulation, and introduce a self-teacher variant for
training from scratch.Where the paper says it
[30] Tianwei Yin, Michaël Gharbi, Richard Zhang, Eli Shechtman, Frédo Durand, William T Freeman, and Taesung Park. One-step diffusion with distribution matching distillation. p. 0 1 Introduction
Diffusion models [9, 25] and flow matching models [16, 17] have achieved remarkable progress in
high-fidelity generation, while relying on multi-step sampling. To reduce generation cost, prior work
has focused on few-step generation through consistency-based flow-map models [1, 6, 12, 24] and
distribution matching distillation (DMD) [29, 30].Where the paper says it
[31] Kaiwen Zheng, Yuji Wang, Qianli Ma, Huayu Chen, Jintao Zhang, Yogesh Balaji, Jianfei Chen, Ming-Yu Liu, Jun Zhu, and Qinsheng Zhang. Large scale diffusion distillation via score-regularized continuous-time consistency. p. 2 Additionally, it exhibits mode-seeking behavior and is known to reduce the
diversity of generated samples compared to forward divergence [19]. Building upon this, [2, 5, 31] incorporate both consistency-based forward divergence and DMDbased backward divergence to cover both mode-covering and mode-seeking behavior, and have been
successfully deployed on large-scale models.Where the paper says it
[33] A theoretical analysis A.1 Score-Velocity Connection
[34] A theoretical analysis A.1 Score-Velocity Connection
[37] Flow Map Matching as a Flow Map Model and Consistency Model
[38] Flow Map Matching as a Flow Map Model and Consistency Model
[39] Deriving FSF-DMD
[40] A theoretical analysis A.3 Deriving FSF-DMD
[47] Algorithm 1 FSF-DMD distillation training
[48] Algorithm 1 FSF-DMD distillation training
[49] Algorithm 1 FSF-DMD distillation training
[57] Algorithm 1 FSF-DMD distillation training
[58] Algorithm 1 FSF-DMD distillation training
[43] A theoretical analysis A.3 Deriving FSF-DMD
[44] A theoretical analysis A.3 Deriving FSF-DMD
[45] A theoretical analysis A.3 Deriving FSF-DMD
[46] A theoretical analysis A.3 Deriving FSF-DMD
[47] Algorithm 1 FSF-DMD distillation training
[48] Algorithm 1 FSF-DMD distillation training
[49] Algorithm 1 FSF-DMD distillation training
[57] Algorithm 1 FSF-DMD distillation training
[58] Algorithm 1 FSF-DMD distillation training
[43] A theoretical analysis A.3 Deriving FSF-DMD
[44] A theoretical analysis A.3 Deriving FSF-DMD
[45] A theoretical analysis A.3 Deriving FSF-DMD
[46] A theoretical analysis A.3 Deriving FSF-DMD
[47] Algorithm 1 FSF-DMD distillation training
[48] Algorithm 1 FSF-DMD distillation training
[49] Algorithm 1 FSF-DMD distillation training
[57] Algorithm 1 FSF-DMD distillation training
[58] Algorithm 1 FSF-DMD distillation training
FSF-DMD suggests that flow-map generators can supply a useful fake-score correction in DMD while preserving the distribution-matching perspective. The method keeps the DMD view that few-step generators benefit from a teacher-fake distributional correction, but replaces the auxiliary fake-score estimator with a generator-induced surrogate. p. 8 5 Conclusion
We presented FSF-DMD, a fake-score-network-free DMD formulation for flow-map generators. The
method keeps the DMD view that few-step generators benefit from a teacher-fake distributional
correction, but replaces the auxiliary fake-score estimator with a generator-induced surrogate. The
method keeps the DMD view that few-step generators benefit from a teacher-fake distributional
correction, but replaces the auxiliary fake-score estimator with a generator-induced surrogate. This
frames the flow-map structure not only as a forward-divergence training mechanism, but also as a
way to provide the reverse-divergence signal without an explicit fake-score network. The resulting
objective extends to flow-map-consistent backward simulation and to a self-teacher variant for training
from scratch. The experiments support the effectiveness of this formulation in the studied settings.Where the paper says it
Where the paper says it
Where the paper says it
Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.
Improvements for AI systems
-
Fake-score-network-free DMD (FSF-DMD) can be applied to flow-map generators by replacing the auxiliary fake-score estimator with a generator induced surrogate, which is
a generator induced surrogate.
This allows thegenerator itself to supply the reverse divergence signal
without requiring an additional network. -
FSF-DMD extends its formulation to backward simulation for flow maps and a self-teacher variant for training from scratch, enabling it to be used in settings where previous methods required separate trackers or explicit fake-velocity objectives.
-
In the flow-map initialized setting, FSF-DMD improves the consistency distillation baseline from 6.06 to 3.85 FID and reaches its best FID within 4K steps, earlier than listed DMD2 comparisons without an explicit fake-score network or adversarial branch.
-
The method achieves better generative performance in the flow-map initialized setting by computing the fake velocity as
the endpoint pseudo-velocity Ftheta(ˆxt;t, 0)
under the PF-ODE coupling, which is derived from the generator's structure. -
For training from scratch, FSF-DMD can be applied to train flow-map models entirely from scratch without a teacher network, suggesting that
this reverse-divergence surrogate can improve the flow-map generator in Sec. 4.4.
Abstract
Distribution Matching Distillation (DMD) provides an effective distribution-level correction for few-step generation, while relying on an auxiliary fake-score network to track the evolving generative distribution. Recent work combines DMD with a flow-map objective to exploit the benefits of both methods: mode-seeking and mode-covering behaviors. Since this fake-score network increases memory and update overhead, we study whether this explicit tracker can be avoided when the generator has a flow-map structure. In this work, we propose Fake-Score-network-Free DMD (FSF-DMD), a DMD-style correction for flow-map generators, replacing the auxiliary fake-score estimator with a generator-induced surrogate. We observe that this surrogate can be derived from the definition of the fake velocity under a certain perturbation assumption. Building on this insight, we derive a DMD-style objective without a fake-side network, extend it to backward simulation for flow maps, and to a self-teacher variant for training from scratch. In our ImageNet-1K 256 times 256 experiments, FSF-DMD is competitive with the studied DMD baselines when initialized from pretrained networks and when trained from scratch.
Related papers
- TEA: Text Encoder Alignment for Robust Concept Erasure in Text-to-Image Models
- Do Egocentric Video-Language Models Capture Both Hand- and Object-Centric Cues?
- Aura: Consistent Multi-Subject Video Generation via VLM-Grounded Semantic Alignment
- PVRF: All-in-one Adverse Weather Removal via Prior-modulated and Velocity-constrained Rectified Flow
- DLWM: Diverse Latent World Models for Efficient Multimodal Reasoning
- Out of Sight, Still in Mind: Token Compression for Omni-LLMs