Introspective Coupling: Self-Explanation Training Tracks Behavioral Change Despite Fixed Supervision
cs.CL, cs.AI, cs.LG
Submitted: 2026-06-30
Updated: 2026-11-08
Comments: Accepted to NeurIPS 2026. 28 pages, 22 figures, 2 tables
Project page: https://time-for-consistency.github.io
License: http://creativecommons.org/licenses/by/4.0/
The gist: When does training language models (LMs) to generate explanations of their predictions yield faithful introspection…
Terminology
Abstract
When does training language models (LMs) to generate explanations of their predictions yield faithful introspection, rather than superficial imitation? We study LMs trained to explain which features of their inputs influenced their behavior, using models' counterfactual behavior on modified inputs as supervision. Surprisingly, we find that LMs trained on fixed counterfactual explanations derived from their earlier checkpoints, or even from behaviorally similar models in different families, frequently produce explanations more faithful to their own current behaviors than to those of their training targets. This "introspective" coupling between LM explanations and behaviors occurs when training explanations remain sufficiently correlated with current behaviors over the course of training, even as behaviors themselves shift. We also show that introspective coupling tracks behavior shifts: when explanation training is provided concurrently with other post-training objectives, explanations change to match without requiring updated supervision. This phenomenon appears in multiple tasks, including sycophancy and refusal, and is robust to label noise. Overall, our results show that even fixed datasets of counterfactual explanations can provide scalable and generalizable post-training signal for introspection.
Sources
- Reasoning Models Don't Always Say What They Think
- Does It Make Sense to Speak of Introspection in Large Language Models?
- Learning to Interpret Weight Differences in Language Models
- The Llama 3 Herd of Models
- Monitoring Monitorability
- Counterfactual Simulation Training for Chain-of-Thought Faithfulness
- Predictive Concept Decoders: Training Scalable End-to-End Interpretability Assistants
- Training LLMs for Honesty via Confessions
- Chain of Thought Monitorability: A New and Fragile Opportunity for AI Safety
- Measuring Faithfulness in Chain-of-Thought Reasoning
- Emergent Introspection in AI is Content-Agnostic
- Training Language Models to Explain Their Own Computations
- Mechanisms of Introspective Awareness
- Latent Introspection: Models Can Detect Prior Concept Injections
- Self-Interpretability: LLMs Can Describe Complex Internal Processes that Drive Their Decisions
- Can LLMs Introspect? A Reality Check
- Simple synthetic data reduces sycophancy in large language models
- Qwen3 Technical Report
Related papers
- ALARM: Audio-Language Alignment for Reasoning Models
- ThinkSum: Probabilistic reasoning over sets using large language models
- Coherence boosting: When your pretrained language model is not paying enough attention
- From Facts to Insights: A Persona-Driven Dual Memory Framework and Dataset for Role-Playing Agents
- Improving General Role-Playing Agents via Psychology-Grounded Reasoning and Role-Aware Policy Optimization
- The Art of Asking: Multilingual Prompt Optimization for Synthetic Data