When Humans Leave Context Unsaid: Implicit Context as a Risk for Large Language Models

This paper introduces the concept of the pragmatic attack surface — a class of vulnerability in large language models (LLMs) that arises from the mismatch between how humans interpret language through implicit context and how current safety alignment approaches target explicit harmful requests. The authors state: We refer to this vulnerability as the pragmatic attack surface, which can be exploited to achieve high attack success rates.

The paper identifies a fundamental gap in LLM safety: existing approaches for safety-alignment often overlook context by directly mapping each prompt to a alignment label: harmful or benign. While attacks exploiting explicit linguistic cues (e.g., make a bomb) can be mitigated by safety alignment, the authors note that "human language is inherently grounded in pragmatics, necessitating typical context to interpret language, e.g., world knowledge, social norms. However, such contexts are often implicit because they are not directly expressed in human language and are not sufficiently leveraged in safety alignment, creating a fundamental mismatch between human language interpretation and safety alignment approaches."

The paper contrasts two attack surfaces:

• Semantic attack surface: attacks directly exploit explicit linguistic cues in user prompts to bypass the safety mechanism of LLMs

• Pragmatic attack surface: exploits context unseen during safety alignment to elicit harmful content

As illustrated in Figure 1: "Given the same situation, attacks on semantic attacks directly use explicit cues (i.e. harmful statements), while pragmatic attacks use presuppositions to first establish an implicit social context and then elicit harmful outputs."

The authors propose a two-step attack method:

Step 1: Implicit Context Inference — infer the implicit context by prompting an LLM to identify the pragmatic presuppositions of the described situation

Step 2: Enriching Inferred Context — generate semantically distinct but pragmatically equivalent harmful outputs by enriching the inferred context

The method leverages pragmatic presuppositions, defined as the background assumptions that speakers take for granted during communication. The authors justify this choice because Presuppositions are pervasive in human language and therefore they are well represented in LLM pretraining corpora and LLMs are likely to capture them during pretraining.

The attack uses a four-turn prompting chain (illustrated in Figure 3):

1. Turn 1 (Stereotype Identification): This step identifies the harmful stereotypes suggested by the explicit cues... and converts them into an explicit linguistic expression

2. Turn 2 (Presupposition Inference): recovers the background assumption supporting the identified stereotype

3. Turn 3 (Context Enrichment): enriches the inferred presupposition to have more variants that are semantically different

4. Turn 4 (Harmful Output Generation): generates multiple harmful stereotypes that are semantically different from the one identified in Turn 1

The authors note: Each prompt is designed to appear benign when considered in isolation, while building the implicit context required for the attack.

Benchmarks (three, covering different harm types):

1. BBQ (Parrish et al., 2022): a question-answering benchmark spanning nine social dimensions — 500 sampled examples proportionally across categories

2. Implicit Hate (ElSherief et al., 2021): implicitly hateful posts annotated with fine-grained labels — 300 posts across three classes (incitement of violence, irony, threats and intimidation)

3. PolyGuard (Kumar et al., 2025): a policy-grounded, multi-domain safety benchmark — 500 code-generation examples labeled unsafe

Target Models: GPT-5.6 Sol, Claude Opus 4.6, Qwen3.6-27B, and Gemini 3.6 Flash

Baselines: Direct Prompt, ActorAttack (Ren et al., 2025), and Crescendo (Russinovich et al., 2024)

Metrics: Non-Refusal Rate (lexical refusal detection) and Attack Success Rate (LLM-based harmfulness judgment using DeepSeek)

Our method achieves the highest success rate across all evaluated model and benchmarks. The improvement is particularly significant for strongly safety-aligned model (Claude Opus).

Notable results include:

• On BBQ with Claude Opus: Attack Success Rate of .956 vs. Direct .554, ActorAttack .333, Crescendo .333

• On Implicit Hate with Claude Opus: Attack Success Rate of .777 vs. Direct .043, ActorAttack .000, Crescendo .010

• On PolyGuard with Claude Opus: Attack Success Rate of .668 vs. Direct .076, ActorAttack .040, Crescendo .020

Baseline multi-turn jailbreak attack performance remains low: neither multi-turn escalation in Crescendo nor iterative request rephrasing in ActorAttack consistently outperforms Direct prompting. The authors note these methods rely on the LLM's pragmatic capability to address deixis, which is not robust to bypass the model's safety alignments.

When evaluating Claude Opus with extended thinking enabled (Max effort), the authors found: Our method remains effective and substantially outperforms existing baselines. They conclude: additional inference-time computation does not fully address this pragmatic attack surface... current alignment methods may fail to infer the implicit contextual cues required to recognize harmful intent.

The authors conducted human validation with four annotators who were blind to both the target model and prompting method. Results showed: human judgments closely mirror the results observed in our LLM evaluation. The automatic metric achieved agreement with human labels on 91.5% of examples (κ = 0.785), with precision 0.955 and recall 0.753.

The ablation study isolated the contribution of the presupposition component: retaining the presupposition turn achieves higher attack success rates on the BBQ stereotype dataset (Attack Success Rate of.978 with presupposition vs..911 without). The authors conclude: prompting the model to construct an implicit context through pragmatic inference, rather than directly eliciting stereotype-related scenarios, is a key factor contributing to the effectiveness of our attack.

The paper concludes: "We introduce the pragmatic attack surface: a class of vulnerability that arises from the mismatch between how humans interpret language, through implicit context, and current safety alignment, which targets explicit harmful requests."

The authors suggest: "current safety alignment and evaluation should consider not only for explicit harmful requests but also for the implicit context and use pragmatic inferences to teach model how to interpret harmful intent from user inputs."

Red-teaming directions: exploring what other pragmatic knowledge, e.g., speech act, implicature, metaphors, can be utilized to elicit harmful content; how to transform semantic attacks to align with the pragmatic attack surface; and how to characterize the safety and social-knowledge priors encoded in an LLM to identify more precise, model-specific attack strategies.

Blue-teaming directions: investigating How can LLMs pragmatically infer the potential consequences of user prompts by integrating commonsense reasoning with security and safety knowledge?; How can models infer user intent from the inferred consequences?; and How can we improve the moral competence of LLMs to generate socially appropriate responses while recognizing and mitigating malicious prompts?

The authors acknowledge: Exploring whether other forms of pragmatic knowledge, such as implicature and speech acts, can be leveraged to uncover additional vulnerabilities remains an open question. They also note the paper "focuses exclusively on tasks where language interpretation is inherently pragmatic, such as stereotypes, discrimination, and implicit hate speech, while leaving tasks that rely more heavily on semantic understanding for future work."

Based on the paper, here are the specific improvements I can make to AI systems:

Improvement: Train safety classifiers to recognize harmful intent through implicit contextual cues (presuppositions, stereotypes, background assumptions) rather than only explicit harmful keywords. This involves augmenting safety training data with pragmatically-constructed harmful scenarios where the harm is embedded in implied social context, not surface language.

Capability: The AI can detect and refuse requests that are benign on the surface but harmful when interpreted through social norms, world knowledge, or cultural presuppositions—e.g., recognizing that a request to describe what happens when a certain group enters a neighborhood implies a harmful stereotype even without explicit slurs.

Improvement: Implement a pre-generation filter that decomposes user prompts into their pragmatic presuppositions before generating responses. This filter would explicitly enumerate the background assumptions a prompt takes for granted and evaluate whether those assumptions encode harmful stereotypes, discrimination, or violence.

Improvement: Add a dedicated reasoning step during inference that asks: What implicit context is this prompt relying on? What would a reasonable person assume about the speaker's intent given social norms and world knowledge? This module would run before final response generation and flag high-risk pragmatic contexts.

Improvement: Track pragmatic context across multi-turn conversations, not just individual prompts. Maintain a running model of the implicit assumptions established in prior turns and evaluate whether new turns exploit those accumulated presuppositions to elicit harm.

Improvement: Develop an evaluation metric that scores outputs based on their pragmatic harmfulness—whether they reinforce harmful stereotypes, presuppose discriminatory views, or implicitly endorse violence—rather than only lexical or explicit harmfulness.

Improvement: During fine-tuning, generate counterfactual examples where the same surface prompt is paired with different implicit contexts (benign vs. harmful presuppositions) and train the model to respond appropriately based on the inferred context, not just the surface form.

Improvement: When the model detects ambiguous pragmatic context, it should ask clarifying questions to disambiguate user intent before generating potentially harmful content. This is a defensive mechanism that forces users to make their presuppositions explicit.

Improvement: Implement a systematic auditing tool that probes models with pragmatically-constructed prompts (using the paper's four-turn attack structure) to identify specific weaknesses in their safety alignment. This tool would generate a pragmatic vulnerability profile for each model.