Negation Neglect: When models fail to learn negations in training
cs.CL, cs.AI, cs.LG
Submitted: 2026-05-13
Updated: 2026-11-06
Code: https://github.com/UKGovernmentBEIS/inspect_ai
Project page: https://mathai-iclr.github.io/papers/papers/MATHAI_29_paper.pdf
License: http://creativecommons.org/licenses/by/4.0/
The gist: We introduce Negation Neglect, where finetuning LLMs on documents that flag a claim as false makes them believe the claim is true.
Terminology
Abstract
We introduce Negation Neglect, where finetuning LLMs on documents that flag a claim as false makes them believe the claim is true. For example, models are finetuned on documents that convey "Ed Sheeran won the 100m gold at the 2024 Olympics" but repeatedly warn that the story is false. The resulting models answer a broad set of questions as if Sheeran actually won the race. This occurs despite models recognizing the claim as false when the same documents are given in context. In experiments with Qwen3.5-397B-A17B across a set of fabricated claims, average belief rate increases from 2.5% to 88.6% when finetuning on negated documents, compared to 92.4% on documents without negations. Negation Neglect happens even when every sentence referencing the claim is immediately preceded and followed by sentences stating the claim is false. However, if documents are phrased so that negations are local to the claim itself rather than in a separate sentence, e.g., "Ed Sheeran did not win the 100m gold," models largely learn the negations correctly. Negation Neglect occurs in all models tested, including Kimi K2.5, GPT-4.1, and Qwen3.5-35B-A3B. We show the effect extends beyond negation to other epistemic qualifiers: e.g., claims labeled as fictional are learned as if they were true. It also extends beyond factual claims to model behaviors. Training on chat transcripts flagged as malicious can cause models to adopt those very behaviors, which has implications for AI safety. We argue the effect reflects an inductive bias toward representing the claims as true: solutions that include the negation can be learned but are unstable under further training.
Sources
- Lessons from Studying Two-Hop Latent Reasoning
- Taken out of context: On measuring situational awareness in LLMs
- Weird Generalization and Inductive Backdoors: New Ways to Corrupt LLMs
- Alignment faking in large language models
- Model Spec Midtraining: Improving How Alignment Training Generalizes
- Frontier LLMs Still Struggle with Simple Reasoning Tasks
- OpenAI GPT-5 System Card
- EQ-Bench: An Emotional Intelligence Benchmark for Large Language Models
- Just as Humans Need Vaccines, So Do Models: Model Immunization to Combat Falsehoods
- Believe It or Not: How Deeply do LLMs Believe Implanted Facts?
- Olmo 3
- Alignment Pretraining: AI Discourse Causes Self-Fulfilling (Mis)alignment
- Measuring short-form factuality in large language models
- Qwen3 Technical Report
Related papers
- ALARM: Audio-Language Alignment for Reasoning Models
- ThinkSum: Probabilistic reasoning over sets using large language models
- Coherence boosting: When your pretrained language model is not paying enough attention
- From Facts to Insights: A Persona-Driven Dual Memory Framework and Dataset for Role-Playing Agents
- Improving General Role-Playing Agents via Psychology-Grounded Reasoning and Role-Aware Policy Optimization
- The Art of Asking: Multilingual Prompt Optimization for Synthetic Data