Reasoning Errors Have a Region and a Direction in the Residual-Stream Trajectory of LLMs
Hamed Damirchi, Ignacio Meza De la Jara, Damith Ranasinghe, Yuhang Liu, Javen Shi
cs.LG, cs.CL
Submitted: 2026-08-06
License: http://creativecommons.org/licenses/by/4.0/
The gist: As language models are increasingly used for tasks that require verifiable reasoning, reliably distinguishing sound reasoning from flawed reasoning has become an important practical problem.
Terminology
Abstract
As language models are increasingly used for tasks that require verifiable reasoning, reliably distinguishing sound reasoning from flawed reasoning has become an important practical problem. Recent trajectory-based methods seek this signal in layerwise residual-stream displacements, which capture how representations change while attenuating some stable, token-specific information. However, displacement omits the state from which an update originates, whereas restoring the full state risks reintroducing shortcut-prone information. We identify this trade-off and propose a three-stream detector that combines motion with two restricted views of location. A coarse region reader based on vector quantization and a fine direction reader over normalized multi-layer states. This design restores enough state context to interpret the motion without returning to full-state probing. On reasoning benchmarks unseen during training, our method improves selection accuracy by up to 12% over the displacement-only state of the art and 21% over single-layer probing baselines. Although trained only on reasoning benchmarks, it also reads factual completion and fact verification, ahead of every detector we compare against, which places the signal on correctness rather than on a kind of reasoning. Ablations further show that motion, region, and direction provide complementary signals. These results suggest that reasoning validity is better read from state-conditioned motion than from either static states or decontextualized trajectories alone.
Sources
- Eliciting Latent Predictions from Transformers with the Tuned Lens
- Discovering Latent Knowledge in Language Models Without Supervision
- Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge
- The Open Proof Corpus: A Large-Scale Study of LLM-Generated Mathematical Proofs
- How to think step-by-step: A mechanistic understanding of chain-of-thought reasoning
- Toy Models of Superposition
- Transformer Dynamics: A neuroscientific approach to interpretability of large language models
- The emergence of clusters in self-attention dynamics
- The Llama 3 Herd of Models
- I Predict Therefore I Am: Is Next Token Prediction Enough to Learn Human-Interpretable Concepts from Data?
- Understanding and Improving Transformer From a Multi-Particle Dynamic System Point of View
- The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasets
- Language Models are Injective and Hence Invertible
- The Linear Representation Hypothesis and the Geometry of Large Language Models
- Large Language Models are overconfident and amplify human bias
- Steering Language Models With Activation Engineering
- Qwen3 Technical Report
- Qwen2.5 Technical Report
- The Geometry of Reasoning: Flowing Logics in Representation Space
Related papers
- Polynomial-Augmented Neural Networks (PANNs) with Weak Orthogonality Constraints for Enhanced Function and PDE Approximation
- AIRL-S: Unifying Reinforcement Learning and Search-Based Test-Time Scaling via Adversarial Inverse Reinforcement Learning
- Transformers as Bayesian In-Context Experimenters: Smoothness-Adaptive Efficient ATE Estimation
- Convergence issues in Relational Concept Analysis based on AOC-posets
- Beliefs Beyond Posteriors: The Predictive Cost of ELBO Binding
- Understanding Diffusion Models via Ratio-Based Function Approximation with SignReLU Networks