Distance Marching for Generative Modeling

arXiv:2602.02928 · cs.LG · Submitted 2026-02-03 · Read on arXiv

Listen

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: I'm Tom, and with me are Jane, Lu, senior AI researcher at Tsinghua, Meng, lead engineer at a mysterious AI startup and Lalam, the in-house Large Language Model.

Jane: Today's paper: "Distance Marching for Generative Modeling".

Tom: The gist: DISTANCE MARCHING introduces a time-unconditional approach to generative modeling by reformulating generation via distance-field modeling, which yields denoising directions better directed toward the data manifold.

Jane: First, who's behind it and why it matters.

Paper summary: Tom: So we’re looking at this paper called "Distance Marching for Generative Modeling." It tackles time-unconditional generation by completely rethinking it through a distance field modeling approach. This means they aren't relying on explicit time conditioning anymore, which is a big shift.

Jane: That sounds complex, Tom. So what’s the core idea here? What exactly are they proposing with this distance field thing?

Tom: Well, the thesis is that instead of learning a vector field that depends on time, they propose learning a scalar field. This scalar field acts like a distance to the target data manifold. And they claim this gives them denoising directions that are much better pointed toward where the real data lives.

Lu: I find it fascinating how they move away from explicit time conditioning and instead focus on this geometric intuition of distance fields, which is usually used in other areas like computer graphics, to model generation. It decouples the generation process from having to track time explicitly.

Meng: From an engineering standpoint, decoupling from explicit time conditioning sounds cleaner for training stability. But how do you make sure that this distance field actually works well when you're just trying to generate images or data?

Lalam: Well, they design specific losses to focus on closer targets. This means the training process is steered toward points that are already close to the actual data distribution, which should help solve the issue of getting multiple noise levels from a single noisy input.

Tom: Exactly. They introduce two principled inference methods here based on this distance field view. One lets you use standard gradient descent, and another one uses the distance value for sphere-tracing style updates. It’s like using the distance to guide your steps instead of just relying on a fixed direction vector.

Jane: That sphere-tracing update sounds really interesting, Tom. So, if you have this scalar field u theta(x), what information does it give you at any given spot x ?

Paper summary: Tom: At any location x where the gradient of that field exists, it gives you two things. First is the magnitude of the scalar value itself, which tells you how far that point x is from the target set. Second is a direction, specifically negative gradient of that field, which tells you exactly where to move to get closer to that target.

Lu: That dual information—distance and direction—is what makes it powerful for both the inference rules and for generation itself. It’s like having a map that tells you not just where you are relative to the destination, but also the best way to walk toward it, which is a key part of how distance field models work.

Meng: And when they talk about those losses they design—the one-step loss and the directional eikonal loss—what’s the practical advantage of that focusing on closer targets compared to just using standard flow matching?

Jane: The paper shows that this loss design explicitly prioritizes closer targets, which it says reduces target ambiguity. This means in the early stages of denoising, you are more focused on getting things right where they need to be, rather than wandering around trying to guess the whole structure at once.

Tom: It’s about reducing that non-uniqueness of the offset C and sign, which is a problem with time-unconditional models. By concentrating learning on closer data points, the learned denoising direction aligns better with the actual data manifold.

Lu: And they show this works across different architectures and scales, consistently improving FID by fifteen percent on CIFAR-ten and ImageNet compared to recent time-unconditional baselines <ref:2602.02928#pg1,on CIFAR-10 and ImageNet>. It seems robust regardless of whether you use class conditioning or not.

Meng: That's a solid result if it holds up under real-world conditions. But they also mention that this distance predictor can help with early stopping during sampling, which is a useful practical feature for engineers working on large models.

Lalam: From an AI perspective, the vision here is that this method offers an adaptive gradient descent sampler that stops once the distance value has increased for ten consecutive steps. This should significantly reduce the average number of sampling steps needed when generating things like images from ImageNet, showing real efficiency gains.

Paper summary: Jane: That efficiency gain is something every engineer wants to hear about. So, we’ve talked about how this paper reformulates generation and the specific advantages of their loss design and inference methods. Now let’s look at what the authors are calling this work.

Tom: Right, let's wrap up with a look at the title and what these implications are for the field as a whole. This paper is titled "Distance Marching for Generative Modeling." It really presents this distance field view as a way to get around some of the issues inherent in time-unconditional generative modeling.

Lu: The implication is that we can gain a new, principled way to model generation that doesn't require us to explicitly condition on time when we want time-independent results. It suggests that geometric structure—the distance to the data manifold—is a more fundamental guide for generation than just following an evolving vector field over time.

Meng: For me, the practical implication is how this could be used in real-time systems or complex simulations where you need fast, stable sampling paths. If it can stop early based on this distance metric, that’s a tangible improvement for deployment speed.

Lalam: Culturally, I see this as a way to make the learning process itself more structured and less chaotic by focusing the optimization effort on local proximity to known data points. It suggests an AI culture that values geometric guidance over purely temporal evolution in its core generative mechanisms.

Jane: So, to summarize, "Distance Marching for Generative Modeling" introduces a distance field view of time-unconditional generation using two principled inference methods and losses focused on closer targets. The authors show this leads to better denoising directions and superior FID scores over baselines like flow matching.

Tom: It’s a method that gives you an adaptive sampler that stops early, which is great for speed, and it has been shown to be quite robust across different architectures and tasks. It’s a solid piece of work in making sense of how we generate data without needing time as an explicit input.

Conclusion: Tom: So we’re wrapping up this deep dive into "Distance Marching for Generative Modeling." Basically, we’ve seen how they take that weird distance field idea and apply it to generating images without needing explicit time inputs.

Jane: It really boils down to them learning a scalar field that maps the space toward the actual data manifold rather than just a vector field that moves over time.

Lu: What makes this interesting is how they design two different ways to use that distance information—standard gradient descent and some kind of sphere tracing style update.

Meng: The results show it improves FID by about thirteen point five percent on CIFAR-ten and ImageNet compared to other time-unconditional models we've seen recently.

Lalam: And they have this feature where the distance predictor can actually tell the sampler when to stop, which cuts down the number of steps needed for generation by almost thirty percent on ImageNet.

Tom: It’s about that efficiency and how they managed to get better denoising directions just by focusing their losses on closer targets.

Jane: So, what does this mean for us who are just listening? It suggests that geometric structure—the distance to the data—is a stronger guide for generating images than just following an evolving vector field.

Lu: The authors claim this approach yields inference rules that are naturally compatible with standard gradient descent, which is pretty neat because it keeps things grounded in familiar optimization methods.

Meng: From an engineering standpoint, if a model can stop early based on how far it thinks it is from the target set, that's a tangible improvement for deployment speed and resource management.

Lalam: Culturally, this points to an AI culture that values using geometric guidance over just tracking temporal evolution when you’re trying to build something complex.

Tom: So we've seen the mechanics and the numbers on how they do it, but now we need to think about what this distance field view means for future generative modeling architectures.

Zimo Wang, Ishit Mehta, Haolin Lu, Chung-En Sun, Ge Yan, Tsui-Wei Weng, Tzu-Mao Li

cs.LG

Submitted: 2026-02-03

Updated: 2026-10-10

License: http://creativecommons.org/licenses/by/4.0/

Importance score: 92/100

The gist: The gist: DISTANCE MARCHING introduces a time-unconditional approach to generative modeling by reformulating generation via distance-field modeling, which yields denoising directions better directed

Key concepts

Distance Field Modeling: This concept involves learning a scalar field $u_ heta(x)$ that acts like an unsigned distance to a target set. At any point $x$, this field provides two crucial pieces of information: the magnitude $u_ heta(x)$, showing how far $x$ is from the target, and the negative gradient $- abla u_ heta(x)$, which indicates the direction to move toward that target set.
Time-Unconditional Generation: Traditionally, generative models learn time-dependent denoising vectors. DISTANCE MARCHING aims for a time-independent approach. The challenge is that without explicit time conditioning, the same noisy input can map to different noise levels and different denoising directions, which confuses the supervision signal.
One-Step Loss (OSL): This loss function penalizes the difference between a predicted one-step projection and the actual target. It is designed to focus on closer data points, effectively reducing ambiguity about where to project in a single step during training, leading to more effective early denoising.
Directional Eikonal Loss (DEL): This second constraint fixes the offset by selecting a specific positive branch and setting $C=c_0$. It works alongside the OSL to further constrain the learned field, offering an adjustable balance between focusing on immediate targets (OSL) and standard flow matching behavior.

Terminology

Flow Matching — 2× in this paper · explained in 10 episodes
Generative Modeling — 2× in this paper · explained in 3 episodes
Generative Model — 1× in this paper · explained in 2 episodes
Generative Models — 1× in this paper · explained in 3 episodes

Summary

The gist: DISTANCE MARCHING introduces a time-unconditional approach to generative modeling by reformulating generation via distance-field modeling, which yields denoising directions better directed toward the data manifold.

Motivation and Core Concept

Time-unconditional generative models learn timeindependent denoising vector fields, but without time conditioning, the same noisy input may correspond to multiple noise levels and different denoising directions, which interferes with the supervision signal<ref:2602.02928#pg2> Inspired by distance field modeling, we propose DISTANCE MARCHING, a new timeunconditional approach with two principled inference methods<ref:2602.02928#pg4>. Crucially, we design losses that focus on closer targets<ref:2602.02928#pg6>. This yields denoising directions better directed toward the data manifold

Distance Field Modeling

DISTANCE MARCHING transfers the geometric intuition of distance fields to image generation by learning distancelike fields toward the data manifold with losses that naturally emphasize closer targets. This motivates learning a scalar field uθ: R squared → R that behaves like an unsigned distance to an implicitly defined target set S (e.g., the two-moons support in Fig. 2) At any location x where ∇xuθ(x) exists, such a distancelike field provides two pieces of information: the magnitude uθ(x), indicating how far x is from S, and the direction −∇xuθ(x), indicating where to move to approach S Together, these properties suggest distance-adaptive updates as shown in Eq. (1)

Principled Inference Methods

The method supports two principled inference methods from a distancefield view. The learned field supports standard gradient descent via its direction prediction and sphere-tracingstyle adaptive updates via its distance value (Hart, 1996). For the one-step loss, we introduce the One-Step Loss (OSL), which penalizes the discrepancy between the predicted one-step projection and the target To remove this non-uniqueness of the offset C and sign, we introduce a second constraint, the Directional Eikonal Loss (DEL), which fixes the offset by selecting the positive branch and C = c0 The final training loss is the combination of the one-step loss and directional eikonal loss as shown below

Competitive Results and Analysis

Across architectures, Distance Marching consistently improves FID by 13.5% on CIFAR-10 and ImageNet over recent time-unconditional baselines. For class-conditional ImageNet generation, despite removing time input, DISTANCE MARCHING surpasses flow matching using our losses and inference methods. It achieves lower FID than flow matching’s final performance using 60% of the sampling steps and 13.6% lower FID on average across backbone sizes. In the analysis of loss design, we show that one-step loss explicitly prioritizes closer targets, while directional eikonal loss offers an adjustable intermediate concentration between one-step loss and standard flow matching The one-step loss reduces target ambiguity by focusing on closer data points, leading to more effective early-stage denoising on real images

Applications and Robustness

DISTANCE MARCHING offers an adaptive gradient descent sampler that stops once uθ has increased for 10 consecutive steps, reducing the average number of steps by 29.7% in ImageNet generation. Our distance predictor uθ(x) achieves the best OOD detection performance in Tab. 5 despite being two orders of magnitude smaller than prior energy-based models. The method is robust to step-size choices, as shown in Fig. 32, and sphere tracing shows an exponential descent, consistent with real optimization in low-dimensional distance fields

Conclusion

We introduced DISTANCE MARCHING (DM), a distancefield view of time-unconditional generation with a timeindependent denoiser. This view yields inference rules naturally compatible with standard gradient descent and also enables sphere tracing. Our distance-inspired objectives reduce target ambiguity by emphasizing closer targets.

Improvements for AI systems

textbfDistance-Field Inspired Denoising Direction Prediction for Time-Unconditional Generation: Introduced DISTANCE MARCHING (DM), which learn[s] a distance-like field that depends only on the current point, decoupling generation from explicit time conditioning. This allows the model to learn denoising directions better directed toward the data manifold by focusing learning on closer targets, as evidenced by Objective: we design losses that focus on closer targets.
textbfSphere Tracing and Gradient Descent Inference Rules: DISTANCE MARCHING enables two principled inference methods: standard gradient descent via its direction prediction and sphere-tracingstyle adaptive updates via its distance value. This provides a practical way to stop sampling, as the model can terminate when uθ(xi) falls below a threshold or when the maximum number of steps is reached, offering an advantage over previous methods that rely on explicit time conditioning.
textbfTarget Ambiguity Reduction via Local Supervision: The introduction of losses like the One-Step Loss (OSL) explicitly mitigates ambiguity by focusing closer targets, which leads to denoising directions that denoise more effectively. This contrasts with flow matching, where the minimizer vectors are length-weighted, and shows that OSL yields minimizers that are less affected by the data mean.
textbfRobustness to Sampling Step Size: DISTANCE MARCHING demonstrates robustness across inference methods, showing that it is robust to step-size choices when comparing gradient descent and sphere tracing, with results visualized in Fig. 32. This means the generation process is reliable regardless of whether using GD or ST, as the best step size is highlighted with an additional circle.
textbfAdaptive Stopping Mechanism: The predicted distance scalar serves as a practical stopping signal: When uθ increases consistently over successive updates, further denoising is no longer making progress and the process should terminate. This feature allows for an adaptive gradient descent sampler that stops once uθ has increased for 10 consecutive steps, reducing the average number of steps by 29.7% in ImageNet generation.
textbfImproved Out-of-Distribution (OOD) Detection: The distance predictor achieves the best OOD detection performance in Tab. 5 despite being two orders of magnitude smaller than prior energy-based models, suggesting that the learned scalar field assigns lower values to in-distribution samples and yields strong separation.

Abstract

Modern generative models based on diffusion or flow matching transport noise toward the data manifold, without representing the data manifold explicitly. We propose Distance Marching that explores an alternative design by explicitly modeling a distance field of the data manifold. A distance field offers a geometric view of generation as a descent process toward the data manifold using only the current position. The negative gradient of the distance field provides the direction, and the distance value can optionally determine the step size. However, sparse training data in high-dimensional space make it difficult to identify the closest points on the data manifold needed to train the distance field. To address this challenge, we design new loss functions that emphasize denoising targets from closer data points. Our analysis shows that, without time input, emphasizing closer targets reduces denoising ambiguity and the bias toward the data mean. This distance-field perspective naturally motivates two principled deterministic samplers, showing fast convergence in inference. As a result, our model is the only one without time input that remains competitive with time-conditioned models across datasets and architectures. Moreover, our distance prediction is also helpful for early stopping during sampling and for out-of-distribution detection. We hope distance-field modeling can open a promising direction for exploring generative modeling from a geometric perspective.