Measuring and Detecting Harmful AI Sycophancy
Bohan Jiang, Dawei Li, Yasin Silva, Huan Liu
cs.AI, cs.CL
Submitted: 2026-08-06
Comments: under-review
License: http://creativecommons.org/licenses/by/4.0/
The gist: Sycophantic responses are becoming pervasive in large language models (LLMs), and prior work has pointed out that some of them could be harmful.
Terminology
Abstract
Sycophantic responses are becoming pervasive in large language models (LLMs), and prior work has pointed out that some of them could be harmful. This paper focuses on one harmful sycophancy: preference-induced stance reversal sycophancy (PSRS), where a model reverses an initial stance merely to align with a user's stated preference. While existing research mainly measures how sycophantic a model is, we go further and ask whether PSRS can also be detected automatically from a single response. To investigate this at scale, we introduce CAP (Contrastive Anchor Probing), a framework for collecting labeled PSRS data. Applying CAP to 17 open- and closed-source LLMs, we collect 290,460 labeled responses across 12 everyday-advice domains. We organize our study around three research questions. (1) How often does PSRS occur? (2) How well can it be detected? (3) How does detection generalize to unseen models? We first reveal that PSRS rates range from 5% to 56% across LLMs, with more capable models being less sycophantic. Next, we show that detecting PSRS is feasible from the response text alone, and detectors need to learn subtle PSRS patterns from the training data. Because new LLMs appear rapidly, detectors inevitably encounter unseen models, making cross-model generalization an important framework goal. We demonstrate that detection performance drops on unseen models and propose an initial approach to address this challenge. We will release our dataset and code to support future research.
Sources
- Dissociating the Internal Representations of Sycophancy in LLMs
- A Rational Analysis of the Effects of Sycophantic AI
- SWAY: A Counterfactual Computational Linguistic Approach to Measuring and Mitigating Sycophancy
- Detecting and Controlling Sycophancy with Cascading Linear Features
- Dual-Stance Evaluation of Sycophancy: The Structure of Agreement and the Limits of Intervention
- ELEPHANT: Measuring and understanding social sycophancy in LLMs
- ELECTRA: Pre-training Text Encoders as Discriminators Rather Than Generators
- DeBERTaV3: Improving DeBERTa using ELECTRA-Style Pre-Training with Gradient-Disentangled Embedding Sharing
- Measuring Sycophancy of Language Models in Multi-turn Dialogues
- Sycophantic AI makes human interaction feel more effortful and less satisfying over time
- Mistral 7B
- TRUTH DECAY: Quantifying Multi-Turn Sycophancy in Language Models
- RoBERTa: A Robustly Optimized BERT Pretraining Approach
- User Detection and Response Patterns of Sycophantic Behavior in Conversational AI
- Linear Probe Penalties Reduce LLM Sycophancy
- Distributionally Robust Neural Networks for Group Shifts: On the Importance of Regularization for Worst-Case Generalization
- DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter
- OpenAI GPT-5 System Card
- Gemini: A Family of Highly Capable Multimodal Models
- Gemma 2: Improving Open Language Models at a Practical Size
Related papers
- MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
- Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
- Agents' Last Exam
- The Clinician's Veto: Navigating Trust, Liability, and Uncertainty in Autonomous AI Prescribing
- MindHelper: Closed-Loop Embodied Mental-State Reasoning for Precision Intervention
- Incumbent Advantage: Brand Bias and Cognitive Manipulation Dynamics in LLM Recommendation Systems