Gumbel Straight Flow: Distilling Autoregressive Models into One-step Flow Maps
cs.LG, cs.AI
Submitted: 2026-09-30
Updated: 2026-09-30
Project page: http://skylion007.github.io/OpenWebTextCorpus
Sources
- Posterior Refinement: Fast Language Generation via Any-Order Flow Maps
- One Billion Word Benchmark for Measuring Progress in Statistical Language Modeling
- Language Modeling with Hyperspherical Flows
- Continuous diffusion for categorical data
- TinyStories: How Small Can Language Models Be and Still Speak Coherent English?
- Beyond Single Tokens: Distilling Discrete Diffusion Models via Discrete MMD
- ELF: Embedded Language Flows
- Adam: A Method for Stochastic Optimization
- Flow Map Language Models: One-step Language Modeling via Continuous Denoising
- Multisample Flow Matching: Straightening Flows with Minibatch Couplings
- Discrete Flow Maps
- Categorical Flow Maps
- Fast Transformer Decoding: One Write-Head is All You Need
- Self-conditioned Flow Map Language Models via Fixed-point Flows
Related papers
- Polynomial-Augmented Neural Networks (PANNs) with Weak Orthogonality Constraints for Enhanced Function and PDE Approximation
- AIRL-S: Unifying Reinforcement Learning and Search-Based Test-Time Scaling via Adversarial Inverse Reinforcement Learning
- Transformers as Bayesian In-Context Experimenters: Smoothness-Adaptive Efficient ATE Estimation
- Convergence issues in Relational Concept Analysis based on AOC-posets
- Beliefs Beyond Posteriors: The Predictive Cost of ELBO Binding
- Understanding Diffusion Models via Ratio-Based Function Approximation with SignReLU Networks