Unknown Unknowns: Do Hidden Intentions in LLMs Evade Detection?
cs.CL, cs.LG
Submitted: 2026-01-26
Updated: 2026-08-31
Comments: Accepted at EMNLP 2026 (Findings)
License: http://creativecommons.org/licenses/by/4.0/
The gist: LLMs expand accessibility and provide wide-reaching access to information.
Terminology
Abstract
LLMs expand accessibility and provide wide-reaching access to information. Yet these interactions also create opportunities to embed subtle, goal-oriented behaviours that shape what users think and how they behave, a concern reflected in governance frameworks that prohibit manipulative AI. We refer to these behaviours as hidden intentions: covert agendas embedded in a model's outputs that can manipulate users' beliefs and actions. In this work, we examine whether hidden intentions can be identified and characterised, and assess whether detection can serve as a mitigation strategy. To operationalise this, we introduce a social-science-grounded set of ten hidden intention categories and show that they are trivially inducible. A case study further confirms that all ten categories manifest in deployed LLMs. We then evaluate static classifiers and LLM judges on these categories, providing the first systematic analysis of why hidden intentions are difficult to detect. Our stress tests show that, unless false-positive rates are vanishingly small, auditing is dominated by precision-prevalence trade-offs. Capability scaling and reasoning models do not close this gap, suggesting a fundamental challenge for open-world detection. These findings expose a core gap of current AI governance: without new auditing paradigms for open-world, low-prevalence risks, bans on manipulative AI remain difficult to enforce.
Sources
- Taxonomizing Representational Harms using Speech Act Theory
- Rethinking the Evaluation of Secure Code Generation
- Bias and Fairness in Large Language Models: A Survey
- Improving alignment of dialogue agents via targeted human judgements
- TruthEval: A Dataset to Evaluate LLM Truthfulness and Reliability
- Evaluating Large Language Models with Psychometrics
- TRUTH DECAY: Quantifying Multi-Turn Sycophancy in Language Models
- Auditing language models for hidden objectives
- Sociotechnical Harms of Algorithmic Systems: Scoping a Taxonomy for Harm Reduction
- Measuring Stereotype and Deviation Biases in Large Language Models
- Benchmarking Gender and Political Bias in Large Language Models
- The Dark Side of AI Companionship: A Taxonomy of Harmful Algorithmic Behaviors in Human-AI Relationships
Related papers
- Exploring Solution Divergence and Its Effect on Large Language Model Problem Solving
- Ishigaki-IDS-Bench: A Benchmark for Generating Information Delivery Specification from BIM Information Requirements
- Subliminal Steering: Stronger Encoding of Hidden Signals
- MedStruct-S: A Benchmark for Key Discovery, Key-Conditioned QA and Semi-Structured Extraction from OCR Clinical Reports
- The End of Transformers? On Challenging Attention and the Rise of Sub-Quadratic Architectures
- Untangling the Mechanisms of Misleading Context in Medical Question Answering