How Chain-of-Thought Works? Tracing Information Flow from Decoding, Projection, and Activation
cs.AI
Submitted: 2025-07-28
Updated: 2026-08-26
Comments: Accept by ACL 2026
Code: https://github.com/How-Young-X/cot
License: http://creativecommons.org/licenses/by/4.0/
The gist: Chain-of-Thought (CoT) prompting significantly enhances model reasoning, yet its internal mechanisms remain poorly understood.
Terminology
Abstract
Chain-of-Thought (CoT) prompting significantly enhances model reasoning, yet its internal mechanisms remain poorly understood. We analyze CoT's operational principles by reversely tracing information flow across decoding, projection, and activation phases. Our quantitative analysis suggests that CoT may serve as a decoding space pruner, leveraging answer templates to guide output generation, with higher template adherence strongly correlating with improved performance. Furthermore, we surprisingly find that CoT modulates neuron engagement in a task-dependent manner: reducing neuron activation in open-domain tasks, yet increasing it in closed-domain scenarios. These findings offer a novel mechanistic interpretability framework and critical insights for enabling targeted CoT interventions to design more efficient and robust prompts. We released our code and data at https://anonymous.4open.science/r/cot-D247.
Sources
- Contrastive Chain-of-Thought Prompting
- Training Verifiers to Solve Math Word Problems
- How to think step-by-step: A mechanistic understanding of chain-of-thought reasoning
- How Do Multilingual Language Models Remember Facts?
- The Llama 3 Herd of Models
- ROME: Memorization Insights from Text, Logits and Representation
- LLMs Can Easily Learn to Reason from Demonstrations Structure, not content, is what matters!
- Text and Patterns: For Effective Chain of Thought, It Takes Two to Tango
- Concise Thoughts: Impact of Output Length on LLM Reasoning and Cost
- On the Hardness of Faithful Chain-of-Thought Reasoning in Large Language Models
- Gemma 2: Improving Open Language Models at a Practical Size
- Analyzing Chain-of-Thought Prompting in Large Language Models via Gradient-based Feature Attributions
- Towards Faithful Natural Language Explanations: A Study Using Activation Patching in Large Language Models
Related papers
- MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
- Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
- Agents' Last Exam
- The Clinician's Veto: Navigating Trust, Liability, and Uncertainty in Autonomous AI Prescribing
- MindHelper: Closed-Loop Embodied Mental-State Reasoning for Precision Intervention
- Incumbent Advantage: Brand Bias and Cognitive Manipulation Dynamics in LLM Recommendation Systems