Evading Chain-of-Thought Monitoring Through Model Poisoning
Giorgio Severi, Shujaat Mirza, Blake Bullwinkel, Amanda Minnich
cs.CR, cs.AI, cs.LG
Submitted: 2026-08-03
Comments: 15 pages, 2 figures
License: http://creativecommons.org/licenses/by/4.0/
The gist: Chain-of-thought (CoT) monitoring is an increasingly important component of AI safety stacks but relies on the assumption that a model's reasoning trace is informative about its actions.
Terminology
Abstract
Chain-of-thought (CoT) monitoring is an increasingly important component of AI safety stacks but relies on the assumption that a model's reasoning trace is informative about its actions. This work studies the limits of CoT monitoring through the lens of model poisoning. We demonstrate that backdoors can be implanted into reasoning models to elicit an attacker-chosen behavior while their CoT traces appear entirely benign. We find that these CoT-Hidden backdoors can be induced through simple fine-tuning recipes across reasoning-model architectures and sizes. When direct poisoning is ineffective, we introduce a curriculum training approach that progressively teaches the model to produce an attacker-chosen output while concealing the behavior from its reasoning traces. These findings suggest that CoT monitoring may be better framed as a question about the consistency between a model's reasoning trace and its final response than as anomaly detection within a trace. We further examine the mechanisms that allow models to suppress evidence of the target behavior from their reasoning traces. Causal interventions locate a trigger-conditioned activation pathway that does not depend on the visible reasoning, and residual stream verbalizations provide an anomaly warning near answer generation, but do not identify the trigger, target, or backdoor mechanism.
Sources
- Analyzing and Improving Chain-of-Thought Monitorability Through Information Theory
- Chain-of-Thought Reasoning In The Wild Is Not Always Faithful
- Monitoring Reasoning Models for Misbehavior and the Risks of Promoting Obfuscation
- Reasoning Models Don't Always Say What They Think
- Training Verifiers to Solve Math Word Problems
- Monitoring Monitorability
- Sleeper Agents: Training Deceptive LLMs that Persist Through Safety Training
- BeaverTails: Towards Improved Safety Alignment of LLM via a Human-Preference Dataset
- Chain of Thought Monitorability: A New and Fragile Opportunity for AI Safety
- Critical-CoT: A Robust Defense Framework against Reasoning-Level Backdoor Attacks in Large Language Models
- Language Models Don't Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Prompting
- A Survey of Recent Backdoor Attacks and Defenses in Large Language Models
Related papers
- SoK: AI-Augmented Binary Reversing
- Relaxed Sender Anonymity for CBDC Interbank Settlement: A Zero-Knowledge Approach on Permissioned EVM
- Calibration-Family Overfit: Why Trusted Sabotage Monitors Don't Transfer Across Lineages
- Estimating the Social Cost of Corporate Data Breaches
- Efficient Fuzzy PSI under One-Sided Assumptions
- Sealing the Audit-Runtime Gap for LLM Skills