Position: It's Time to Optimize LLMs for Self-Consistency
Itamar Pres, Belinda Z. Li, Laura Ruis, Zifan Carl Guo, Keya Hu, Mehul Damani, Isha Puri, Ekdeep Singh Lubana, Jacob Andreas
cs.CL, cs.AI
Submitted: 2026-07-31
Comments: Accepted at the 43rd International Conference on Machine Learning (ICML 2026), Position Paper Track
License: http://creativecommons.org/licenses/by/4.0/
The gist: Despite ever-increasing sophistication in language model (LM) pre- and post-training pipelines, many important failures persist: models overcondition on user framing ("sycophancy"), exhibit
Terminology
Abstract
Despite ever-increasing sophistication in language model (LM) pre- and post-training pipelines, many important failures persist: models overcondition on user framing ("sycophancy"), exhibit incomplete logical generalization, and produce confident but incorrect responses. We argue that these failures arise from a modeling assumption permeating all aspects of the pipeline: that behavior can be specified and evaluated independently on single-output pairs. Many model failures are difficult, if not impossible, to detect without reasoning about relationships between a model's responses across inputs. In this position paper, we propose self-consistency as a framework for understanding these failures. We first observe that a wide variety of techniques designed to improve specific aspects of LM behavior-targeting properties as diverse as adversarial robustness and factual coherence-can be understood as special cases of a common "consistency optimization" procedure and addressed with a standard set of optimization tools. We next outline a set of new model properties that could be achieved by optimizing for consistency, and conclude with a discussion of what it would mean to develop generally consistent LMs, including the capabilities they would enable and the objections they raise.
Sources
- Looking Inward: Language Models Can Learn About Themselves by Introspection
- Constitutional AI: Harmlessness from AI Feedback
- Reasoning Models Don't Always Say What They Think
- Attacking Large Language Models with Projected Gradient Descent
- Bias-Augmented Consistency Training Reduces Biased Reasoning in Chain-of-Thought
- Learning to Interpret Weight Differences in Language Models
- The Llama 3 Herd of Models
- Alignment faking in large language models
- Sycophancy to Subterfuge: Investigating Reward-Tampering in Large Language Models
- Monitoring Monitorability
- The Personality Illusion: Revealing Dissociation Between Self-Reports & Behavior in LLMs
- Counterfactual Simulation Training for Chain-of-Thought Faithfulness
- Why Language Models Hallucinate
- Beneath the Surface of Consistency: Exploring Cross-lingual Knowledge Representation Sharing in LLMs
- Measuring Faithfulness in Chain-of-Thought Reasoning
- Prompt Injection attack against LLM-integrated Applications
- Training Language Models to Explain Their Own Computations
- DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models
- Self-Interpretability: LLMs Can Describe Complex Internal Processes that Drive Their Decisions
- Proximal Policy Optimization Algorithms
Related papers
- Exploring Solution Divergence and Its Effect on Large Language Model Problem Solving
- Ishigaki-IDS-Bench: A Benchmark for Generating Information Delivery Specification from BIM Information Requirements
- Subliminal Steering: Stronger Encoding of Hidden Signals
- MedStruct-S: A Benchmark for Key Discovery, Key-Conditioned QA and Semi-Structured Extraction from OCR Clinical Reports
- The End of Transformers? On Challenging Attention and the Rise of Sub-Quadratic Architectures
- Untangling the Mechanisms of Misleading Context in Medical Question Answering