Sleight of Word Benchmark: Can Language Models Notice If Their Own Output Was Tampered With?
cs.CL, cs.AI
Submitted: 2026-08-30
Updated: 2026-08-30
Comments: Accepted at INLG 2026
Project page: https://fractalego.github.io/Sleight-of-Word
License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
Terminology
Sources
- Detecting the Disturbance: A Nuanced View of Introspective Abilities in LLMs
- Language Models (Mostly) Know What They Know
- Measuring Faithfulness in Chain-of-Thought Reasoning
- Emergent Introspective Awareness in Large Language Models
- Can Reasoning Models Detect Changes to their Chains of Thought?
- Ignore Previous Prompt: Attack Techniques For Language Models
- Replacing Judges with Juries: Evaluating LLM Generations with a Panel of Diverse Models
- PromptRobust: Towards Evaluating the Robustness of Large Language Models on Adversarial Prompts
Related papers
- Exploring Solution Divergence and Its Effect on Large Language Model Problem Solving
- Ishigaki-IDS-Bench: A Benchmark for Generating Information Delivery Specification from BIM Information Requirements
- Subliminal Steering: Stronger Encoding of Hidden Signals
- MedStruct-S: A Benchmark for Key Discovery, Key-Conditioned QA and Semi-Structured Extraction from OCR Clinical Reports
- The End of Transformers? On Challenging Attention and the Rise of Sub-Quadratic Architectures
- Untangling the Mechanisms of Misleading Context in Medical Question Answering