Eliciting Intrinsic Hallucinations in LLMs via Semantically Equivalent Adversarial Attacks

arXiv:2608.04286 · cs.CL, cs.LG · Submitted 2026-08-04 · Read on arXiv

Atri Vivek Sharma, Brian Formento, Alessio Lomuscio

cs.CL, cs.LG

Submitted: 2026-08-04

Comments: To be presented at COLM 2026

Code: https://github.com/atriviveksharma/intrinsic_hall

License: http://creativecommons.org/licenses/by/4.0/

The gist: Large language models (LLMs) are often used in conjunction with external knowledge sources to improve their factual accuracy and decrease hallucinations, through methods such as Retrieval-Augmented

Terminology

Abstract

Large language models (LLMs) are often used in conjunction with external knowledge sources to improve their factual accuracy and decrease hallucinations, through methods such as Retrieval-Augmented Generation (RAG). However, these systems remain susceptible to intrinsic hallucinations, where the model generates unfaithful or fabricated information that is not supported by the retrieved evidence. We propose a novel framework to assess model robustness against this phenomenon by stress-testing using natural, semantically equivalent variations of a user query found via adversarial optimization methods. We apply our framework, which enforces strict semantic equivalence constraints and an intrinsic hallucination objective, to a range of adversarial attack techniques across white-box, gray-box, and black-box adversarial settings. Evaluating these attacks on 5 open-source and 5 closed-source generator models across 3 datasets, we demonstrate that even state-of-the-art models are highly susceptible to meaning-preserving perturbations, which significantly degrade contextual faithfulness (by up to 50% for GPT-5-mini). Our findings indicate that faithful use of in-context evidence remains fragile even in state-of-the-art LLMs, motivating architectures and training objectives that enforce robust grounding independent of surface query form. Code is available at: https://github.com/atriviveksharma/intrinsic hall

Sources

Related papers