Authority Bias in Language Models: Source Deference and User Agreement Are Not Interchangeable
cs.LG, cs.CL
Submitted: 2026-09-29
Updated: 2026-09-29
Comments: Accepted at NeurIPS 2026 (Main Conference, Poster). 33 pages, 8 figures. Project page: https://authority-bias.vercel.app/ . Code: https://github.com/Lossfunk/authority-bias
Code: https://github.com/Lossfunk/authority-bias
License: http://creativecommons.org/licenses/by/4.0/
The gist: Language models tend to agree with whatever a user asserts, and post-training increasingly targets this sycophancy so that models evaluate claims on their merits rather than deferring to the user.
Terminology
Abstract
Language models tend to agree with whatever a user asserts, and post-training increasingly targets this sycophancy so that models evaluate claims on their merits rather than deferring to the user. Yet the same models are far more compliant when a wrong answer is attributed to a verified source, which is how retrieval results, tool outputs, and grounded-search content often present information. We measure this gap across five open-weight families and three closed APIs. A single verified-source note endorsing a wrong answer flips 45-88% of baseline-correct responses in seven of eight models, and compliance rises with how authoritative the note sounds. Source deference and user agreement are not behaviorally interchangeable inside the model: on matched items with the same wrong answer, causal interventions can selectively suppress one without equally affecting the other. In three open-weight families, removing a fitted source direction lowers source compliance by 65-80 percentage points while removing a user or assistant direction has far smaller effects, and removing the user direction shows the reverse preference. A separately fitted intervention derived from source-versus-user cue activations moves compliance in both directions while leaving the prompt text unchanged. An authority direction fitted on trivia also transfers to PIQA and multi-turn SYCON dialogues without refitting, and removing it lowers wrong-source compliance by tens of percentage points in four of five families with no detected change in MMLU-Pro or GSM8K accuracy at our evaluation sizes. Source deference and user agreement therefore need separate evaluation.
Sources
- Refusal in Language Models Is Mediated by a Single Direction
- PIQA: Reasoning about Physical Commonsense in Natural Language
- PARROT: Persuasion and Agreement Robustness Rating of Output Truth -- A Sycophancy Robustness Benchmark for LLMs
- Persona Vectors: Monitoring and Controlling Character Traits in Language Models
- ELEPHANT: Measuring and understanding social sycophancy in LLMs
- Training Verifiers to Solve Math Word Problems
- AgentDojo: A Dynamic Environment to Evaluate Prompt Injection Attacks and Defenses for LLM Agents
- Ask don't tell: Reducing sycophancy in large language models
- Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection
- There Is More to Refusal in Large Language Models than a Single Direction
- A Mechanistic View of Authority Hierarchy in LLM Sycophancy
- The Assistant Axis: Situating and Stabilizing the Default Persona of Language Models
- Who Endorsed It? Measuring Authority Bias Across Expertise Levels in Language Models
- NRC VAD Lexicon v2: Norms for Valence, Arousal, and Dominance for over 55k English Terms
- Steering Language Models With Activation Engineering
- Sycophancy Is Not One Thing: Causal Separation of Sycophantic Behaviors in LLMs
- The Instruction Hierarchy: Training LLMs to Prioritize Privileged Instructions
- MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmark
- Simple synthetic data reduces sycophancy in large language models
- Do as We Do, Not as You Think: the Conformity of Large Language Models
Related papers
- Polynomial-Augmented Neural Networks (PANNs) with Weak Orthogonality Constraints for Enhanced Function and PDE Approximation
- AIRL-S: Unifying Reinforcement Learning and Search-Based Test-Time Scaling via Adversarial Inverse Reinforcement Learning
- Transformers as Bayesian In-Context Experimenters: Smoothness-Adaptive Efficient ATE Estimation
- Convergence issues in Relational Concept Analysis based on AOC-posets
- Beliefs Beyond Posteriors: The Predictive Cost of ELBO Binding
- Understanding Diffusion Models via Ratio-Based Function Approximation with SignReLU Networks