Thinking vs. NoThinking: Towards Interpreting Reasoning Mechanisms of Large Language Models via Sparse Autoencoders
Bo Cheng, Qiaolin Lu, Yi Chang, Yuan Wu
cs.CL
Submitted: 2026-08-08
Updated: 2026-08-11
License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
Terminology
Sources
- DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning
- How does Chain of Thought Think? Mechanistic Interpretability of Chain-of-Thought Reasoning with Sparse Autoencoding
- Progress measures for grokking via mechanistic interpretability
- Emergent World Representations: Exploring a Sequence Model Trained on a Synthetic Task
- Representation Engineering: A Top-Down Approach to AI Transparency
- Improving Dictionary Learning with Gated Sparse Autoencoders
- Sparse Autoencoders Find Highly Interpretable Features in Language Models
- Feature Extraction and Steering for Enhanced Chain-of-Thought Reasoning in Language Models
- Scaling and evaluating sparse autoencoders
- Gemma Scope: Open Sparse Autoencoders Everywhere All At Once on Gemma 2
- SAEs Are Good for Steering -- If You Select the Right Features
- Decoding Dense Embeddings: Sparse Autoencoders for Interpreting and Discretizing Dense Retrieval
- k-Sparse Autoencoders
- DeepMath-103K: A Large-Scale, Challenging, Decontaminated, and Verifiable Mathematical Dataset for Advancing Reasoning
- Reasoning Models Can Be Effective Without Thinking
- Adam: A Method for Stochastic Optimization
Related papers
- Exploring Solution Divergence and Its Effect on Large Language Model Problem Solving
- Ishigaki-IDS-Bench: A Benchmark for Generating Information Delivery Specification from BIM Information Requirements
- Subliminal Steering: Stronger Encoding of Hidden Signals
- MedStruct-S: A Benchmark for Key Discovery, Key-Conditioned QA and Semi-Structured Extraction from OCR Clinical Reports
- The End of Transformers? On Challenging Attention and the Rise of Sub-Quadratic Architectures
- Untangling the Mechanisms of Misleading Context in Medical Question Answering