When Decodability Is Not Enough: Logical Validity Representations, Behavioral Dissociation, and Causal Tests in Language Models
The paper examines a gap between an LLM's internal logic and its actual output behavior. Researchers found that while models hold logical validity internally (high decodability), they fail to express this knowledge reliably in their answers. This suggests accuracy alone is not enough for trustworthy AI; we must verify internal mechanisms.