A Step in the Right Place: Understanding and Optimizing Timestep Placement in Continuous Diffusion Language Models
cs.CL
Submitted: 2026-10-11
Updated: 2026-10-11
Comments: 24 pages, 7 figures. Reproducibility materials: https://github.com/collapsar-z/StepTune-reproducibility
Code: https://github.com/collapsar-z/StepTune-reproducibility
License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
Sources
- Structured Denoising Diffusion Models in Discrete State-Spaces
- On the Trajectory Regularity of ODE-based Diffusion Sampling
- LangFlow: Continuous Diffusion Rivals Discrete in Language Modeling
- Continuous diffusion for categorical data
- S4S: Solving for a Diffusion Model Solver
- ELF: Embedded Language Flows
- Adaptive Reparametrized Time for Score-Based Diffusion Sampling
- Adaptive Second-Order Solvers for Fast Stochastic Diffusion Sampling
- Diffusion-LM Improves Controllable Text Generation
- AURORA-LM: Autoencoding Unified Representation for Continuous-Latent Diffusion Language Modeling
- Optimal Stepsize for Diffusion Sampling
- Align Your Steps: Optimizing Sampling Schedules in Diffusion Models
- Bespoke Non-Stationary Solvers for Fast Sampling of Diffusion and Flow Models
- Learning to Discretize Denoising Diffusion ODEs
- Learning Fast Samplers for Diffusion Models by Differentiating Through Sample Quality
- Continuous Diffusion Scales Competitively with Discrete Diffusion for Language
- Few-Step Diffusion Sampling Through Instance-Aware Discretizations
Related papers
- Exploring Solution Divergence and Its Effect on Large Language Model Problem Solving
- Ishigaki-IDS-Bench: A Benchmark for Generating Information Delivery Specification from BIM Information Requirements
- Subliminal Steering: Stronger Encoding of Hidden Signals
- MedStruct-S: A Benchmark for Key Discovery, Key-Conditioned QA and Semi-Structured Extraction from OCR Clinical Reports
- The End of Transformers? On Challenging Attention and the Rise of Sub-Quadratic Architectures
- Untangling the Mechanisms of Misleading Context in Medical Question Answering