Kernel-Gradient Drifting Models
cs.LG, math.DG
Submitted: 2026-05-11
Updated: 2026-11-02
License: http://creativecommons.org/licenses/by/4.0/
The gist: We propose kernel-gradient drifting, a one-step generative modeling framework that replaces the fixed Euclidean displacement direction in drifting models with directions induced by the kernel itself.
Terminology
Abstract
We propose kernel-gradient drifting, a one-step generative modeling framework that replaces the fixed Euclidean displacement direction in drifting models with directions induced by the kernel itself. Standard drifting is attractive because it enables fast, high-quality generation without distilling a large pretrained diffusion model, but its theory is currently understood mainly for Gaussian kernels, where the drift coincides with smoothed score matching and is identifiable. Our gradient-based reformulation exposes this score-based structure for general kernels: the resulting drift is the score difference between kernel-smoothed data and model distributions, yielding identifiability for characteristic kernels and a smoothed-KL descent interpretation of the smoothed drifting dynamics. Since kernel gradients are intrinsic tangent vectors, the same construction extends naturally to Riemannian manifolds and to discrete data via the Fisher-Rao geometry of the probability simplex. Across spherical geospatial data, promoter DNA and molecule generation, kernel-gradient drifting enables state-of-the-art one-step generation beyond the Euclidean setting without distillation. Code is available here.
Sources
- Kernel smoothing on manifolds
- Gradient Flow Drifting: Generative Modeling via Wasserstein Gradient Flows of KDE-Approximated Divergences
- Generalised Flow Maps for Few-Step Generative Modelling on Riemannian Manifolds
- Generative Modeling via Drifting
- Continuous diffusion for categorical data
- Drifting Fields are not Conservative
- Action de groupe sur la compactification hybride
- Diff-Instruct: A Universal Approach for Transferring Knowledge From Pre-trained Diffusion Models
- Categorical Flow Maps
- Universal kernels via harmonic analysis on Riemannian symmetric spaces
- Generative Drifting is Secretly Score Matching: a Spectral and Variational Perspective
- Riemannian MeanFlow
Related papers
- A Physics-Regulated Neural Framework for Learning 3D Grain Growth Dynamics
- FlexMoE: Learning One-for-All Multi-Budget Expert Pruning for Flexible MoE Deployment
- OptiMind: Improving Small Language Models for Optimization Modeling through Formulation Guidance and Multi-Turn Learning
- Staying in the Sweet Spot: Boost Reasoning Evolution via Responsive Rollout Difficulty Manipulation
- Examining the Impact of Data and Training Methods on Emergent and Subliminal Misalignment
- Rethinking Black-Box On-Policy Distillation: A Rubric-based Analysis