Generative Recursive Reasoning
cs.AI
Submitted: 2026-05-19
Updated: 2026-10-15
Code: https://github.com/cloneofsimo/d3pm
Project page: https://ahn-ml.github.io/gram-website
License: http://creativecommons.org/licenses/by/4.0/
The gist: How should future neural reasoning systems implement extended computation? Recursive Reasoning Models (RRMs) offer a promising alternative to autoregressive sequence extension by performing iterative
Terminology
Abstract
How should future neural reasoning systems implement extended computation? Recursive Reasoning Models (RRMs) offer a promising alternative to autoregressive sequence extension by performing iterative latent-state refinement with shared transition functions. Yet existing RRMs are largely deterministic, following a single latent trajectory and converging to a single prediction. We introduce Generative Recursive reAsoning Models (GRAM), a framework that turns recursive latent reasoning into probabilistic multi-trajectory computation. GRAM models reasoning as a stochastic latent trajectory, enabling multiple hypotheses, alternative solution strategies, and inference-time scaling through both recursive depth and parallel trajectory sampling. This yields a latent-variable generative model supporting conditional reasoning via p θ(y x) and, with fixed or absent inputs, unconditional generation via p θ(x). Trained with amortized variational inference, GRAM improves over deterministic recurrent and recursive baselines on structured reasoning and multi-solution constraint satisfaction tasks, while demonstrating an unconditional generation capability. https://ahn-ml.github.io/gram-website
Sources
- Training Large Language Models to Reason in a Continuous Latent Space
- Reasoning by Superposition: A Theoretical Perspective on Chain of Continuous Thought
- Continuous Chain of Thought Enables Parallel Exploration and Reasoning
- Looped Transformers are Better at Learning Learning Algorithms
- Hierarchical Reasoning Model
- Less is More: Recursive Reasoning with Tiny Networks
- Universal Transformers
- The Consciousness Prior
- On the Measure of Intelligence
- ARC-AGI-2: A New Challenge for Frontier AI Reasoning Systems
- Unbiasing Truncated Backpropagation Through Time
- Scaling up Test-Time Compute with Latent Reasoning: A Recurrent Depth Approach
- Self-Consistency Improves Chain of Thought Reasoning in Language Models
- Text Generation Beyond Discrete Token Sampling
- Soft Thinking: Unlocking the Reasoning Potential of LLMs in Continuous Concept Space
- Soft Tokens, Hard Truths
- CODI: Compressing Chain-of-Thought into Continuous Space via Self-Distillation
- Hybrid Latent Reasoning via Reinforcement Learning
- Mixture-of-Recursions: Learning Dynamic Recursive Depths for Adaptive Token-Level Computation
- CoTFormer: A Chain-of-Thought Driven Architecture with Budget-Adaptive Computation Cost at Inference
Related papers
- MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
- Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
- Agents' Last Exam
- The Clinician's Veto: Navigating Trust, Liability, and Uncertainty in Autonomous AI Prescribing
- MindHelper: Closed-Loop Embodied Mental-State Reasoning for Precision Intervention
- Incumbent Advantage: Brand Bias and Cognitive Manipulation Dynamics in LLM Recommendation Systems