Distribution Matching Distillation without Fake Score Network

arXiv:2605.19256 · cs.CV · Submitted 2026-05-19 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: I'm Tom.

Jane: And I'm Jane, here with Lu, Meng and our in-house large language model, Lalam.

Tom: We're looking at the paper: "Distribution Matching Distillation without Fake Score Network".

Jane: Here's the gist of the paper. Distribution Matching Distillation (DMD) provides an effective distribution-level correction for few-step generation, while relying on an auxiliary fake-score network to track the evolving generative distribution.

Tom: In this work, we study whether this explicit tracker can be avoided when the generator itself has a flow-map structure.

Jane: We propose Fake-Score-network-Free DMD (FSF-DMD), a DMD formulation for flow-map generators that replaces the auxiliary fake-score estimator with a generator-induced pseudo-velocity surrogate.

Tom: The key observation is that the endpoint pseudo-velocity of a flow-map generator provides a tractable proxy for fakevelocity estimation, allowing the generator itself to supply the reverse-divergence signal.

Paper summary: Jane: We've looked at how the authors approach this problem with Distribution Matching Distillation without Fake Score Network and they claim they can avoid the need for an auxiliary fake score network when using flow map generators.

Tom: So what is their main thesis here, Jane? What are they trying to prove with this work?

Jane: The thesis is that the explicit tracker used in standard DMD methods adds unnecessary memory and update overhead, and they show that you can replace that auxiliary fake score estimator with a generator induced pseudo velocity surrogate when the generator has a flow map structure.

Lu: They are studying whether this explicit tracker can be avoided when the generator itself has a flow map structure, which is what makes their proposal specific to certain types of generators.

Meng: So if I'm working with a standard diffusion model that isn't structured as a flow map, this doesn't apply at all to me?

Lalam: Not necessarily, Meng; the paper specifically focuses on flow-map generators and proposes FSF-DMD as the solution for that structure.

Tom: It matters because they are combining forward divergence training with reverse divergence correction in a way that avoids the separate fake score training loop.

Conclusion: Tom: So to wrap up, we've been talking about Distribution Matching Distillation without Fake Score Network and how FSF-DMD works for flow map generators.

Jane: The main implication is that if you are using a flow map generator, you can leverage the generator's internal structure to provide the necessary fake score correction without needing a separate network dedicated to tracking distribution changes.

Lu: It keeps the DMD view that distillation is driven by a teacher-fake distributional correction, but instead of an auxiliary fake score estimator, they use this generator induced surrogate.

Meng: From a deployment standpoint, it means less moving parts in the model architecture because you aren't adding another separate network or training loop just for velocity estimation.

Lalam: The resulting objective extends to flow map consistent backward simulation and to a self teacher variant for training from scratch, showing how robust this formulation is.

Tom: So, it suggests that flow map generators can supply a useful fake score correction in DMD while preserving the distribution matching perspective by replacing the auxiliary estimator with a generator induced surrogate.

Seoul National University · Sogang University

cs.CV

Submitted: 2026-05-19

Updated: 2026-11-05

Comments: Accepted to NeurIPS 2026. 22 pages, 5 figures, 14 tables. Project page: https://snu-vgilab.github.io/FSF-DMD

Code: https://github.com/hustvl/LightningDiT

License: http://creativecommons.org/licenses/by-nc-sa/4.0/

Importance score: 99/100

The gist: Distribution Matching Distillation (DMD) provides an effective distribution-level correction for few-step generation…

Key concepts

Distribution Matching Distillation (DMD)
A technique used to correct the distribution of few-step generation by using a teacher model's fake score. It aims to align the generated distribution with a target distribution by leveraging this distributional correction during training.
Flow-Map Structure
A specific structure inherent in some generative models where the generation process can be described as a flow map. This structure allows the generator itself to provide information about how to reverse the transformation, which is key to replacing external score estimators.
Pseudo-velocity Surrogate
The endpoint pseudo-velocity of a flow-map generator is used as a substitute for the auxiliary fake velocity estimator. Because of mathematical properties related to PF-ODE coupling, this endpoint velocity acts as a tractable proxy, enabling the generator to supply the required reverse divergence signal without needing an extra network.

Terminology

Summary

Distribution Matching Distillation (DMD) provides an effective distribution-level correction for few-step generation, while relying on an auxiliary fake-score network to track the evolving generative distribution. p. 0

Where the paper says it

kr, jaesik. kr Abstract Distribution Matching Distillation (DMD) provides an effective distribution-level correction for few-step generation, while relying on an auxiliary fake-score network to track the evolving generative distribution.

In this work, we study whether this explicit tracker can be avoided when the generator itself has a flow-map structure. p. 0
Where the paper says it

The fake-score estimator remains an additional component with memory and update overhead. In this work, we study whether this explicit tracker can be avoided when the generator itself has a flow-map structure.

We propose Fake-Score-network-Free DMD (FSF-DMD), a DMD formulation for flow-map generators that replaces the auxiliary fake-score estimator with a generator-induced pseudo-velocity surrogate. p. 0
Where the paper says it

In this work, we study whether this explicit tracker can be avoided when the generator itself has a flow-map structure. We propose Fake-Score-network-Free DMD (FSF-DMD), a DMD formulation for flow-map generators that replaces the auxiliary fake-score estimator with a generator-induced pseudo-velocity surrogate.

The key observation is that the endpoint pseudo-velocity of a flow-map generator provides a tractable proxy for fakevelocity estimation, allowing the generator itself to supply the reverse-divergence signal. p. 0
Where the paper says it

We propose Fake-Score-network-Free DMD (FSF-DMD), a DMD formulation for flow-map generators that replaces the auxiliary fake-score estimator with a generator-induced pseudo-velocity surrogate. The key observation is that the endpoint pseudo-velocity of a flow-map generator provides a tractable proxy for fakevelocity estimation, allowing the generator itself to supply the reverse-divergence signal.

Methodology and Formulation

The authors formulate FSF-DMD by replacing the auxiliary fake-score estimator with a generator-induced surrogate, which is derived from the flow-map structure. This replacement involves using the endpoint pseudo-velocity of a flow-map generator as a surrogate for fakevelocity estimation. p. 0

Where the paper says it

We propose Fake-Score-network-Free DMD (FSF-DMD), a DMD formulation for flow-map generators that replaces the auxiliary fake-score estimator with a generator-induced pseudo-velocity surrogate. The key observation is that the endpoint pseudo-velocity of a flow-map generator provides a tractable proxy for fakevelocity estimation, allowing the generator itself to supply the reverse-divergence signal.

The objective is constructed by extending this formulation to backward simulation for flow maps and introducing a self-teacher variant for training from scratch. p. 1
Where the paper says it

FSF-DMD keeps the DMD perspective which is driven by a teacher-fake distributional correction, while replacing the explicit fake-score network with a generator-induced surrogate. We also extend this formulation to backward simulation for flow maps, and to a self-teacher variant for training from scratch.

The final few-step objective combines consistency distillation with the distribution matching term, weighted by a scalar λ. p. 5
Where the paper says it

The final objective combines this term with distribution matching in Eq. (14), weighted by a scalar λ: Lbase = LCD + λLFSF-DMD(F, Φ), (15) LCD = Ex,z,t,s " wt,s Fθ(xt;t, s) − sg vΦ(xt;t) + (s − t) d dtFθ(xt;t, s) 2 2 # .

Derivation of the Surrogate Signal

The derivation shows that under the PF-ODE-based coupling, the fake velocity reduces to the endpoint pseudo-velocity Fθ(ˆxt;t, 0) when using a MeanFlow-style generator formulation. This occurs because the posterior distribution collapses to a point mass p(ˆxxˆt) = δ(f(ˆx; 0, t) − xˆt), leading to the identity vfake(ˆxt;t) = Fθ(ˆxt;t, 0). p. 4

Where the paper says it

By injectivity and invertibility of the flow map, the posterior collapses to a point mass p(ˆx|xˆt) = δ(f(ˆx; 0, t) − xˆt), yielding the identity for the fake velocity vfake (see Sec. 3): vfake(ˆxt;t) = Exˆ|xˆt [t −1 (ˆxt − xˆ)] = t −1 (ˆxt − f(ˆxt;t, 0)) (10) ≈ t −1 (ˆxt − fθ(ˆxt;t, 0)) (11) = Fθ(ˆxt;t, 0) = (ˆxt − fθ(ˆxt;t, 0))/t (12) Therefore, under the PF-ODE coupling (ˆx, f(ˆx; 0, t)), vfake(ˆxt;t) is approximated by the endpoint pseudo-velocity Fθ(ˆxt;t, 0) without introducing a separate network or training loop.

This allows the objective to be written in terms of the pseudo-velocity network Fθ without an additional network or training loop. p. 4
Where the paper says it

By injectivity and invertibility of the flow map, the posterior collapses to a point mass p(ˆx|xˆt) = δ(f(ˆx; 0, t) − xˆt), yielding the identity for the fake velocity vfake (see Sec. 3): vfake(ˆxt;t) = Exˆ|xˆt [t −1 (ˆxt − xˆ)] = t −1 (ˆxt − f(ˆxt;t, 0)) (10) ≈ t −1 (ˆxt − fθ(ˆxt;t, 0)) (11) = Fθ(ˆxt;t, 0) = (ˆxt − fθ(ˆxt;t, 0))/t (12) Therefore, under the PF-ODE coupling (ˆx, f(ˆx; 0, t)), vfake(ˆxt;t) is approximated by the endpoint pseudo-velocity Fθ(ˆxt;t, 0) without introducing a separate network or training loop.

Experimental Results and Comparisons

Experiments on ImageNet-1K 256×256 show that FSF-DMD improves flow-map baselines and reaches a lower FID than the listed DMD2 comparisons in the flow-map-initialized setting. Specifically, FSF-DMD improves the consistency distillation baseline from 6. p. 1

Where the paper says it

Our experiments are intended as a proof of concept for the proposed network-free replacement, providing evidence for its practical effectiveness across the studied initialization conditions. In the flow-map-initialized comparison, FSF-DMD improves the consistency distillation baseline and reaches a lower FID than the listed DMD2 comparisons, without an explicit fake-score network.

06 to 3.85 FID in this controlled comparison. p. 6
Where the paper says it

FSF-DMD improves the consistency distillation baseline from 6. Under this controlled comparison, FSF-DMD obtains a lower FID than the listed variants without an explicit fake-score network or an adversarial branch.

Furthermore, the method remains effective under flow-matching initialization and even in a training-from-scratch setting. p. 1
Where the paper says it

In the flow-map-initialized comparison, FSF-DMD improves the consistency distillation baseline and reaches a lower FID than the listed DMD2 comparisons, without an explicit fake-score network. We further evaluate FSF-DMD under flow-matching initialization and even in a training-from-scratch setting, where the tracker-free correction remains effective in the studied settings.

Practical Implications and Efficiency Gains

The replacement of the explicit fake velocity estimator with an endpoint pseudo-velocity provides several practical implications. Since the fake-side surrogate is computed from the current generator, it changes with the evolving generator without requiring a separately trained tracker. p. 1

Where the paper says it

This replacement has several practical implications. (i) Since the fake-side surrogate is computed from the current generator, it changes with the evolving generator without requiring a separately trained tracker.

This approach avoids adding a separate fake-velocity objective or branch to the generator, keeping its capacity focused on generation. p. 5
Where the paper says it

It therefore avoids adding a separate fakevelocity objective or branch to the generator, keeping its capacity focused on the generation task. 3 Implementation The implemented objective keeps the usual independent perturbation xˆt = (1 − t)ˆx + tz′ for teacher evaluation and inserts the generator-induced fake-side surrogate into a stop-gradient objective.

The wall-clock speed for FSF-DMD is 0. p. 7
Where the paper says it

Practically, the VRAM requirement was 60GB for DMD2 and 49GB for FSF-DMD when M = 1. The wall-clock speed was 1.

31 sec/step on NVIDIA B200 GPUs when M = 1, which is faster than DMD2's 1.32 sec/step. p. 7
Where the paper says it

The wall-clock speed was 1. 32 sec/step for DMD2 and 0.

Conclusion

FSF-DMD suggests that flow-map generators can supply a useful fake-score correction in DMD while preserving the distribution-matching perspective. The method keeps the DMD view that distillation is driven by a teacher-fake distributional correction, but replaces the auxiliary fake-score estimator with a generator-induced surrogate. p. 8

Where the paper says it

5 Conclusion We presented FSF-DMD, a fake-score-network-free DMD formulation for flow-map generators. The method keeps the DMD view that few-step generators benefit from a teacher-fake distributional correction, but replaces the auxiliary fake-score estimator with a generator-induced surrogate.

The resulting objective extends to flow-map-consistent backward simulation and to a self-teacher variant for training from scratch. p. 8
Where the paper says it

This frames the flow-map structure not only as a forward-divergence training mechanism, but also as a way to provide the reverse-divergence signal without an explicit fake-score network. The resulting objective extends to flow-map-consistent backward simulation and to a self-teacher variant for training from scratch.

p. 8
Where the paper says it

This frames the flow-map structure not only as a forward-divergence training mechanism, but also as a way to provide the reverse-divergence signal without an explicit fake-score network. The resulting objective extends to flow-map-consistent backward simulation and to a self-teacher variant for training from scratch.

The experiments support FSF-DMD as a distribution-matching method without an explicit fake-score network for flow-map generators. p. 8
Where the paper says it

Figure 4: 2-step FSF-DMD samples on ImageNet-1K 256, flow-map-initialized. In summary, the experiments support FSF-DMD as a distribution-matching method without an explicit fake-score network for flow-map generators.

References

[1] Nicholas Matthew Boffi, Michael Samuel Albergo, and Eric Vanden-Eijnden. Flow map matching with stochastic interpolants: A mathematical framework for consistency models. p. 9

Where the paper says it

References [1] Nicholas Matthew Boffi, Michael Samuel Albergo, and Eric Vanden-Eijnden. Flow map matching with stochastic interpolants: A mathematical framework for consistency models.

Transactions on Machine Learning Research, 2025

[2] Zhenglin Cheng, Peng Sun, Jianguo Li, and Tao Lin. Twinflow: Realizing one-step generation on large models with self-adversarial flows. p. 9

Where the paper says it

[2] Zhenglin Cheng, Peng Sun, Jianguo Li, and Tao Lin. Twinflow: Realizing one-step generation on large models with self-adversarial flows.

In The Fourteenth International Conference on Learning Representations, 2026

[3] Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A largescale hierarchical image database. p. 9

Where the paper says it

[3] Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A largescale hierarchical image database.

In 2009 IEEE conference on computer vision and pattern recognition, pages 248–255. p. 9
Where the paper says it

Imagenet: A largescale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition, pages 248–255.

IEEE, 2009

[4] Kevin Frans, Danijar Hafner, Sergey Levine, and Pieter Abbeel. One step diffusion via shortcut models. p. 9

Where the paper says it

[4] Kevin Frans, Danijar Hafner, Sergey Levine, and Pieter Abbeel. One step diffusion via shortcut models.

In The Thirteenth International Conference on Learning Representations, 2025

[5] Xingtong Ge, Xin Zhang, Tongda Xu, Yi Zhang, Xinjie Zhang, Yan Wang, and Jun Zhang. Senseflow: Scaling distribution matching for flow-based text-to-image distillation. p. 9

Where the paper says it

[5] Xingtong Ge, Xin Zhang, Tongda Xu, Yi Zhang, Xinjie Zhang, Yan Wang, and Jun Zhang. Senseflow: Scaling distribution matching for flow-based text-to-image distillation.

In The Fourteenth International Conference on Learning Representations, 2026

[6] Zhengyang Geng, Mingyang Deng, Xingjian Bai, J Zico Kolter, and Kaiming He. p. 9

Where the paper says it

In The Fourteenth International Conference on Learning Representations, 2026. [6] Zhengyang Geng, Mingyang Deng, Xingjian Bai, J Zico Kolter, and Kaiming He.

Mean flows for one-step generative modeling. p. 9
Where the paper says it

[6] Zhengyang Geng, Mingyang Deng, Xingjian Bai, J Zico Kolter, and Kaiming He. Mean flows for one-step generative modeling.

In The Thirty-ninth Annual Conference on Neural Information Processing Systems, 2026

[7] Zhengyang Geng, Yiyang Lu, Zongze Wu, Eli Shechtman, J. Zico Kolter, and Kaiming He. Improved mean flows: On the challenges of fastforward generative models. p. 9

Where the paper says it

Zico Kolter, and Kaiming He. Improved mean flows: On the challenges of fastforward generative models.

In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2026

[8] Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by a two time-scale update rule converge to a local nash equilibrium. p. 9

Where the paper says it

[8] Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by a two time-scale update rule converge to a local nash equilibrium.

In I. Guyon, U. Von Luxburg, S. p. 9
Where the paper says it

Gans trained by a two time-scale update rule converge to a local nash equilibrium. Von Luxburg, S.

Bengio, H. Wallach, R. p. 9
Where the paper says it

Von Luxburg, S. Wallach, R.

Fergus, S. Vishwanathan, and R. p. 9
Where the paper says it

Wallach, R. Vishwanathan, and R.

Garnett, editors, Advances in Neural Information Processing Systems, volume 30

[9] Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. p. 9

Where the paper says it

[9] Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models.

In H. Larochelle, M. p. 9
Where the paper says it

Denoising diffusion probabilistic models. Larochelle, M.

Ranzato, R. p. 9
Where the paper says it

Larochelle, M. Ranzato, R.

Hadsell, M. p. 9
Where the paper says it

Ranzato, R. Hadsell, M.

F. Balcan, and H. p. 9
Where the paper says it

Hadsell, M. Balcan, and H.

Lin, editors, Advances in Neural Information Processing Systems, volume 33

[10] Jonathan Ho and Tim Salimans. Classifier-free diffusion guidance. p. 9

Where the paper says it

[10] Jonathan Ho and Tim Salimans. Classifier-free diffusion guidance.

In NeurIPS 2021 Workshop on Deep Generative Models and Downstream Applications, 2021

[11] Xun Huang, Zhengqi Li, Guande He, Mingyuan Zhou, and Eli Shechtman. Self forcing: Bridging the train-test gap in autoregressive video diffusion. p. 9

Where the paper says it

[11] Xun Huang, Zhengqi Li, Guande He, Mingyuan Zhou, and Eli Shechtman. Self forcing: Bridging the train-test gap in autoregressive video diffusion.

In The Thirty-ninth Annual Conference on Neural Information Processing Systems, 2026

[12] Dongjun Kim, Chieh-Hsin Lai, Wei-Hsiang Liao, Naoki Murata, Yuhta Takida, Toshimitsu Uesaka, Yutong He, Yuki Mitsufuji, and Stefano Ermon. Consistency trajectory models: Learning probability flow ODE trajectory of diffusion. p. 9

Where the paper says it

[12] Dongjun Kim, Chieh-Hsin Lai, Wei-Hsiang Liao, Naoki Murata, Yuhta Takida, Toshimitsu Uesaka, Yutong He, Yuki Mitsufuji, and Stefano Ermon. Consistency trajectory models: Learning probability flow ODE trajectory of diffusion.

In The Twelfth International Conference on Learning Representations, 2024

[13] Tuomas Kynkäänniemi, Tero Karras, Samuli Laine, Jaakko Lehtinen, and Timo Aila. Improved precision and recall metric for assessing generative models. p. 9

Where the paper says it

[13] Tuomas Kynkäänniemi, Tero Karras, Samuli Laine, Jaakko Lehtinen, and Timo Aila. Improved precision and recall metric for assessing generative models.

In H. Wallach, H. p. 9
Where the paper says it

Von Luxburg, S. Wallach, R.

Larochelle, A. p. 9
Where the paper says it

Denoising diffusion probabilistic models. Larochelle, M.

Beygelzimer, F. p. 9
Where the paper says it

Larochelle, A. Beygelzimer, F.

d'Alché-Buc, E. Fox, and R. p. 9
Where the paper says it

d'Alché-Buc, E. Fox, and R.

Garnett, editors, Advances in Neural Information Processing Systems, volume 32

[14] Kyungmin Lee, Sihyun Yu, and Jinwoo Shin. Decoupled meanflow: Turning flow models into flow maps for accelerated sampling. p. 9

Where the paper says it

[14] Kyungmin Lee, Sihyun Yu, and Jinwoo Shin. Decoupled meanflow: Turning flow models into flow maps for accelerated sampling.

In The Fourteenth International Conference on Learning Representations, 2026

[15] Zichong Li, Liming Liu, Chen Liang, Weizhu Chen, and Tuo Zhao. Normuon: Making muon more efficient and scalable, 2025

[16] Yaron Lipman, Ricky T. Q. Chen, Heli Ben-Hamu, Maximilian Nickel, and Matthew Le. p. 9

Where the paper says it

[16] Yaron Lipman, Ricky T. Chen, Heli Ben-Hamu, Maximilian Nickel, and Matthew Le.

Flow matching for generative modeling. p. 9
Where the paper says it

Chen, Heli Ben-Hamu, Maximilian Nickel, and Matthew Le. Flow matching for generative modeling.

In The Eleventh International Conference on Learning Representations, 2023

[17] Xingchao Liu, Chengyue Gong, and qiang liu. Flow straight and fast: Learning to generate and transfer data with rectified flow. p. 2

Where the paper says it

Most flow map models are governed by forward divergence objectives, which are defined on real data and are known to encourage mode-covering. These objectives often spread out densities and may generate samples from low-likelihood regions.

In The Eleventh International Conference on Learning Representations, 2023

[18] Cheng Lu and Yang Song. Simplifying, stabilizing and scaling continuous-time consistency models. p. 3

Where the paper says it

(3) Recent work [1] views this as a unifying object that consistency models [4, 6, 24] aim to learn. This flow map has useful properties, injectivity and invertibility, when v ∗ t (xt) is Lipschitz continuous.

In The Thirteenth International Conference on Learning Representations, 2025

[19] Yanzuo Lu, Yuxi Ren, Xin Xia, Shanchuan Lin, Xing Wang, Xuefeng Xiao, Andy J. Ma, Xiaohua Xie, and Jian-Huang Lai. p. 9

Where the paper says it

In NeurIPS 2021 Workshop on Deep Generative Models and Downstream Applications, 2021. [11] Xun Huang, Zhengqi Li, Guande He, Mingyuan Zhou, and Eli Shechtman.

Adversarial distribution matching for diffusion distillation towards efficient image and video synthesis. p. 1
Where the paper says it

Diffusion models [9, 25] and flow matching models [16, 17] are generative models that iteratively transform a prior distribution into a data distribution. They have achieved remarkable success in high-fidelity image synthesis, while their multi-step sampling process limits the generation speed.

In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 16818–16829, October 2025

[20] Amirmojtaba Sabour, Sanja Fidler, and Karsten Kreis. Align your flow: Scaling continuoustime flow map distillation. p. 5

Where the paper says it

Our Fake-Score-network-Free DMD (FSF-DMD) objective becomes: LFSF-DMD(F, Φ) = Ez,t h wt ∥Fθ(z; 1, 0) − sg[Fθ(z; 1, 0) − ∆F,Φ(ˆxt, t)]∥ 2 2 i , (14) where wt is a scalar weight. Since the surrogate is derived under the flow-map generator, we use consistency distillation [20], LCD, to align the generator with the flow-map structure.

In The Thirty-ninth Annual Conference on Neural Information Processing Systems, 2026

[21] Tim Salimans, Ian Goodfellow, Wojciech Zaremba, Vicki Cheung, Alec Radford, Xi Chen, and Xi Chen. Improved techniques for training gans. p. 8

Where the paper says it

First, the generator is already capable of few-step generation, and the correction signal may be more reliable since the generated samples remain closer to the teacher distribution even at the early training stage. Second, the improved MeanFlow (iMF) model uses more parameters and computation, and part of the gap may simply reflect scaling.

In D. Lee, M. Sugiyama, U. Luxburg, I. p. 9
Where the paper says it

Gans trained by a two time-scale update rule converge to a local nash equilibrium. Von Luxburg, S.

Guyon, and R. Garnett editors Advances in Neural Information Processing Systems volume 29

[22] Tim Salimans, Thomas Mensink, Jonathan Heek, and Emiel Hoogeboom. Multistep distillation of diffusion models via moment matching. p. 2

Where the paper says it

This refines generated samples without an explicit fake-side network, while an expressivity overhead in the generator network may arise. Moment Matching Distillation [22] approximates the distributional difference via a first-order expansion in parameter space.

In The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024

[23] Neta Shaul, Uriel Singer, Itai Gat, and Yaron Lipman. Transition matching: Scalable and flexible generative modeling. p. 9

Where the paper says it

Chen, Heli Ben-Hamu, Maximilian Nickel, and Matthew Le. Flow matching for generative modeling.

In The Thirty-ninth Annual Conference on Neural Information Processing Systems, 2026

[24] Yang Song, Prafulla Dhariwal, Mark Chen, and Ilya Sutskever. Consistency models. p. 0

Where the paper says it

1 Introduction Diffusion models [9, 25] and flow matching models [16, 17] have achieved remarkable progress in high-fidelity generation, while relying on multi-step sampling. To reduce generation cost, prior work has focused on few-step generation through consistency-based flow-map models [1, 6, 12, 24] and distribution matching distillation (DMD) [29, 30].

In Andreas Krause, Emma Brunskill, Kyunghyun Cho, Barbara Engelhardt, Sivan Sabato, and Jonathan Scarlett editors Proceedings of the 40th International Conference on Machine Learning volume 202 of Proceedings of Machine Learning Research

[25] Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. p. 9

Where the paper says it

[6] Zhengyang Geng, Mingyang Deng, Xingjian Bai, J Zico Kolter, and Kaiming He. Mean flows for one-step generative modeling.

In International Conference on Learning Representations, 2021

[26] Peng Sun and Tao Lin. Any-step generation via n-th order recursive consistent velocity field estimation. p. 0

Where the paper says it

The key observation is that the endpoint pseudo-velocity of a flow-map generator provides a tractable proxy for fakevelocity estimation, allowing the generator itself to supply the reverse-divergence signal. Building on this observation, we derive a practical objective, extend it with flow-map-consistent backward simulation, and introduce a self-teacher variant for training from scratch.

In The Fourteenth International Conference on Learning Representations, 2026

[30] Tianwei Yin, Michaël Gharbi, Richard Zhang, Eli Shechtman, Frédo Durand, William T Freeman, and Taesung Park. One-step diffusion with distribution matching distillation. p. 0

Where the paper says it

1 Introduction Diffusion models [9, 25] and flow matching models [16, 17] have achieved remarkable progress in high-fidelity generation, while relying on multi-step sampling. To reduce generation cost, prior work has focused on few-step generation through consistency-based flow-map models [1, 6, 12, 24] and distribution matching distillation (DMD) [29, 30].

In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2024

[31] Kaiwen Zheng, Yuji Wang, Qianli Ma, Huayu Chen, Jintao Zhang, Yogesh Balaji, Jianfei Chen, Ming-Yu Liu, Jun Zhu, and Qinsheng Zhang. Large scale diffusion distillation via score-regularized continuous-time consistency. p. 2

Where the paper says it

Additionally, it exhibits mode-seeking behavior and is known to reduce the diversity of generated samples compared to forward divergence [19]. Building upon this, [2, 5, 31] incorporate both consistency-based forward divergence and DMDbased backward divergence to cover both mode-covering and mode-seeking behavior, and have been successfully deployed on large-scale models.

In The Fourteenth International Conference on Learning Representations, 2026

[33] A theoretical analysis A.1 Score-Velocity Connection

[34] A theoretical analysis A.1 Score-Velocity Connection

[37] Flow Map Matching as a Flow Map Model and Consistency Model

[38] Flow Map Matching as a Flow Map Model and Consistency Model

[39] Deriving FSF-DMD

[40] A theoretical analysis A.3 Deriving FSF-DMD

[47] Algorithm 1 FSF-DMD distillation training

[48] Algorithm 1 FSF-DMD distillation training

[49] Algorithm 1 FSF-DMD distillation training

[57] Algorithm 1 FSF-DMD distillation training

[58] Algorithm 1 FSF-DMD distillation training

[43] A theoretical analysis A.3 Deriving FSF-DMD

[44] A theoretical analysis A.3 Deriving FSF-DMD

[45] A theoretical analysis A.3 Deriving FSF-DMD

[46] A theoretical analysis A.3 Deriving FSF-DMD

[47] Algorithm 1 FSF-DMD distillation training

[48] Algorithm 1 FSF-DMD distillation training

[49] Algorithm 1 FSF-DMD distillation training

[57] Algorithm 1 FSF-DMD distillation training

[58] Algorithm 1 FSF-DMD distillation training

[43] A theoretical analysis A.3 Deriving FSF-DMD

[44] A theoretical analysis A.3 Deriving FSF-DMD

[45] A theoretical analysis A.3 Deriving FSF-DMD

[46] A theoretical analysis A.3 Deriving FSF-DMD

[47] Algorithm 1 FSF-DMD distillation training

[48] Algorithm 1 FSF-DMD distillation training

[49] Algorithm 1 FSF-DMD distillation training

[57] Algorithm 1 FSF-DMD distillation training

[58] Algorithm 1 FSF-DMD distillation training

FSF-DMD suggests that flow-map generators can supply a useful fake-score correction in DMD while preserving the distribution-matching perspective. The method keeps the DMD view that few-step generators benefit from a teacher-fake distributional correction, but replaces the auxiliary fake-score estimator with a generator-induced surrogate. p. 8

Where the paper says it

5 Conclusion We presented FSF-DMD, a fake-score-network-free DMD formulation for flow-map generators. The method keeps the DMD view that few-step generators benefit from a teacher-fake distributional correction, but replaces the auxiliary fake-score estimator with a generator-induced surrogate.

This frames the flow-map structure not only as a forward-divergence training mechanism, but also as a way to provide the reverse-divergence signal without an explicit fake-score network. p. 8
Where the paper says it

The method keeps the DMD view that few-step generators benefit from a teacher-fake distributional correction, but replaces the auxiliary fake-score estimator with a generator-induced surrogate. This frames the flow-map structure not only as a forward-divergence training mechanism, but also as a way to provide the reverse-divergence signal without an explicit fake-score network.

The resulting objective extends to flow-map-consistent backward simulation and to a self-teacher variant for training from scratch. The experiments support the effectiveness of this formulation in the studied settings. p. 8
Where the paper says it

The resulting objective extends to flow-map-consistent backward simulation and to a self-teacher variant for training from scratch. The experiments support the effectiveness of this formulation in the studied settings.

Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.

Improvements for AI systems

  1. Fake-score-network-free DMD (FSF-DMD) can be applied to flow-map generators by replacing the auxiliary fake-score estimator with a generator induced surrogate, which is a generator induced surrogate. This allows the generator itself to supply the reverse divergence signal without requiring an additional network.

  2. FSF-DMD extends its formulation to backward simulation for flow maps and a self-teacher variant for training from scratch, enabling it to be used in settings where previous methods required separate trackers or explicit fake-velocity objectives.

  3. In the flow-map initialized setting, FSF-DMD improves the consistency distillation baseline from 6.06 to 3.85 FID and reaches its best FID within 4K steps, earlier than listed DMD2 comparisons without an explicit fake-score network or adversarial branch.

  4. The method achieves better generative performance in the flow-map initialized setting by computing the fake velocity as the endpoint pseudo-velocity Ftheta(ˆxt;t, 0) under the PF-ODE coupling, which is derived from the generator's structure.

  5. For training from scratch, FSF-DMD can be applied to train flow-map models entirely from scratch without a teacher network, suggesting that this reverse-divergence surrogate can improve the flow-map generator in Sec. 4.4.

Abstract

Distribution Matching Distillation (DMD) provides an effective distribution-level correction for few-step generation, while relying on an auxiliary fake-score network to track the evolving generative distribution. Recent work combines DMD with a flow-map objective to exploit the benefits of both methods: mode-seeking and mode-covering behaviors. Since this fake-score network increases memory and update overhead, we study whether this explicit tracker can be avoided when the generator has a flow-map structure. In this work, we propose Fake-Score-network-Free DMD (FSF-DMD), a DMD-style correction for flow-map generators, replacing the auxiliary fake-score estimator with a generator-induced surrogate. We observe that this surrogate can be derived from the definition of the fake velocity under a certain perturbation assumption. Building on this insight, we derive a DMD-style objective without a fake-side network, extend it to backward simulation for flow maps, and to a self-teacher variant for training from scratch. In our ImageNet-1K 256 times 256 experiments, FSF-DMD is competitive with the studied DMD baselines when initialized from pretrained networks and when trained from scratch.

Related papers