Next Thoughts Are Distributions: Generative Autoregressive Reasoning in the Latent Space
cs.AI
Submitted: 2026-09-27
Updated: 2026-09-27
Comments: 16 pages
Code: https://github.com/huggingface/open-r1
License: http://creativecommons.org/licenses/by-nc-sa/4.0/
The gist: Reasoning problems often admit multiple valid ways to proceed.
Terminology
Abstract
Reasoning problems often admit multiple valid ways to proceed. Continuous reasoning promises to move computation beyond language tokens into a more compact latent space, but representing several plausible ways to think next remains difficult. We introduce Autoregressive Thought Flow (ATF), which models the next continuous thought as a multimodal distribution. A causal autoregressive model performs the reasoning computation, while a lightweight diffusion head generates a plausible next thought from the resulting condition. The sampled thought is fed back into the model, allowing continuous reasoning to unfold for a variable number of steps while preserving the pretrained backbone. Across mathematical reasoning tasks, ATF improves accuracy with compact latent traces and benefits from reinforcement learning and additional test-time thinking. Multi-sample evaluation shows broader solution coverage, indicating that its multimodal predictions capture useful diversity among reasoning paths. Our results suggest that continuous reasoning is more effective when multiple possible next thoughts remain available rather than being collapsed into a single prediction.
Sources
- Large Concept Models: Language Modeling in a Sentence Representation Space
- Compressed Chain of Thought: Efficient Reasoning Through Dense Representations
- Training Verifiers to Solve Math Word Problems
- Beyond Benchmarks: MathArena as an Evaluation Platform for Mathematics with LLMs
- Causal Diffusion Transformers for Generative Modeling
- Implicit Chain of Thought Reasoning via Knowledge Distillation
- SONAR: Sentence-Level Multimodal and Language-Agnostic Representations
- Think-at-Hard: Dynamic Looped Transformers for Improved Reasoning
- Training Large Language Models to Reason in a Continuous Latent Space
- Classifier-Free Diffusion Guidance
- LaDiR: Latent Diffusion Enhances LLMs for Text Reasoning
- Latent Thought Models with Variational Bayes Inference-Time Computation
- Flow-GRPO: Training Flow Matching Models via Online RL
- AIMO-2 Winning Solution: Building State-of-the-Art Mathematical Reasoning Models with OpenMathReasoning dataset
- Mull-Tokens: Modality-Agnostic Latent Thinking
- Continuous Autoregressive Language Models
- DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models
- CODI: Compressing Chain-of-Thought into Continuous Space via Self-Distillation
- Token Assorted: Mixing Latent and Text Tokens for Improved Language Model Reasoning
- Multimodal Latent Language Modeling with Next-Token Diffusion
Related papers
- MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
- Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
- Agents' Last Exam
- The Clinician's Veto: Navigating Trust, Liability, and Uncertainty in Autonomous AI Prescribing
- MindHelper: Closed-Loop Embodied Mental-State Reasoning for Precision Intervention
- Incumbent Advantage: Brand Bias and Cognitive Manipulation Dynamics in LLM Recommendation Systems