Intrinsic Self-Correction in LLMs: Towards Explainable Prompting via Mechanistic Interpretability
Yu-Ting Lee, Fu-Chieh Chang, Yu-En Shu, Hui-Ying Shih, Pei-Yuan Wu
cs.CL, cs.AI, cs.LG
Submitted: 2026-08-21
Updated: 2026-08-24
Journal ref: 4th Deployable AI Workshop at AAAI 2026
Code: https://github.com/Yu-TingLee/llm-self-correction-mech-interp
License: http://creativecommons.org/licenses/by/4.0/
The gist: Intrinsic moral self-correction refers to the phenomenon where a language model refines its ethical judgments or aligns its outputs purely through prompting.
Terminology
Abstract
Intrinsic moral self-correction refers to the phenomenon where a language model refines its ethical judgments or aligns its outputs purely through prompting. While effective across diverse tasks, its mechanism remains unclear. We hypothesize intrinsic moral self-correction functions by steering hidden representations along interpretable latent directions. Evaluating six LLMs across four morality-related tasks, we demonstrate that the representation shifts induced by self-correction prompts align with contrastive steering vectors. This alignment transfers even when the steering vectors are constructed from a disjoint corpus. Notably, when applied via activation addition, these prompt-induced shifts can alter model behavior more effectively than the self-correction prompts and the steering vectors. Our findings suggest representation steering is the mechanistic driver of intrinsic moral self-correction.
Sources
- On the Intrinsic Self-Correction Capability of LLMs: Uncertainty and Latent Concept
- Confidence Matters: Revisiting Intrinsic Self-Correction Capabilities of Large Language Models
- Steering Language Models With Activation Engineering
- Representation Engineering: A Top-Down Approach to AI Transparency
- Unveiling the Latent Directions of Reflection in Large Language Models
- Mistral 7B
- LFM2 Technical Report
Related papers
- Exploring Solution Divergence and Its Effect on Large Language Model Problem Solving
- Ishigaki-IDS-Bench: A Benchmark for Generating Information Delivery Specification from BIM Information Requirements
- Subliminal Steering: Stronger Encoding of Hidden Signals
- MedStruct-S: A Benchmark for Key Discovery, Key-Conditioned QA and Semi-Structured Extraction from OCR Clinical Reports
- The End of Transformers? On Challenging Attention and the Rise of Sub-Quadratic Architectures
- Untangling the Mechanisms of Misleading Context in Medical Question Answering