Probabilistic Tiny Recursive Model
cs.AI
Submitted: 2026-05-19
Updated: 2026-11-09
License: http://creativecommons.org/licenses/by/4.0/
The gist: Tiny Recursive Models (TRM) solve complex reasoning tasks with a fraction of the parameters of modern large language models (LLMs) by iteratively refining a latent state and final answer.
Terminology
Abstract
Tiny Recursive Models (TRM) solve complex reasoning tasks with a fraction of the parameters of modern large language models (LLMs) by iteratively refining a latent state and final answer. While powerful, their deterministic recursion can lead to convergence at suboptimal solutions, without escape mechanism. A common workaround relies on task-specific input perturbations at test time combined with answer aggregation via voting. We introduce Probabilistic TRM (PTRM), a task-agnostic framework for test-time compute scaling that addresses this limitation through stochastic exploration. PTRM injects Gaussian noise at each deep recursion step, enabling parallel trajectories to explore diverse solution basins, and selects among them using the model's existing Q head (used for early stopping in the original TRM). Without requiring retraining or task-specific augmentations, PTRM enables substantial accuracy gains across benchmarks, including Sudoku-Extreme (87.4% to 98.75%) and on various puzzles from Pencil Puzzle Bench (62.6% to 91.2%). On the latter, PTRM achieves nearly double the accuracy of frontier LLMs (91.2% vs. 55.1%) at less than 0.0001x the cost, using only 7M parameters.
Sources
- Less is More: Recursive Reasoning with Tiny Networks
- Hierarchical Reasoning Model
- On the Measure of Intelligence
- ARC-AGI-2: A New Challenge for Frontier AI Reasoning Systems
- Are Your Reasoning Models Reasoning or Guessing? A Mechanistic Analysis of Hierarchical Reasoning Models
- Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters
- Adaptive Computation Time for Recurrent Neural Networks
- Pencil Puzzle Bench: A Benchmark for Multi-Step Verifiable Reasoning
- Universal Transformers
- Mixture-of-Recursions: Learning Dynamic Recursive Depths for Adaptive Token-Level Computation
- Form Follows Function: Recursive Stem Model
- Training Large Language Models to Reason in a Continuous Latent Space
- Reasoning by Superposition: A Theoretical Perspective on Chain of Continuous Thought
- Measuring Faithfulness in Chain-of-Thought Reasoning
- Reasoning Models Don't Always Say What They Think
Related papers
- Multi-Objective Structured Pruning of LLMs for Latency and Model Size Optimization
- MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
- Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
- Embeddings for Preferences, Not Semantics
- Can We Trust Item Response Theory for AI Evaluation?
- Diagnosing Task Insensitivity in Language Agents