Divergent large language model predictions from convergent representations in ambiguous word pairs
K. Jack Scott, Narun Pat, Veronica Liesaputra
cs.CL
Submitted: 2026-08-03
Comments: 21 main text pages, 20 pages supplemental, 4 figures
Code: https://github.com/scoki211/Divergent_LLM_Predictions_
License: http://creativecommons.org/licenses/by/4.0/
The gist: In this work we investigate how decoder-only transformers resolve lexical ambiguity through layer-by-layer analysis of three models spanning three parameter sizes (GPT-2-Small-117M, Llama-3.2-3B,
Terminology
Abstract
In this work we investigate how decoder-only transformers resolve lexical ambiguity through layer-by-layer analysis of three models spanning three parameter sizes (GPT-2-Small-117M, Llama-3.2-3B, Qwen2.5-32B). For both homonyms and polysemes, we find that representations become maximally distinct in middle layers, then partially reconverge in late layers, while the KL divergence between their next-token predictions reaches its maximum in the final layers. The activation patching experiment provides causal evidence that late-layer representational differences directly determine outputs despite apparent increased similarity in embedding space. Our single-layer ablation experiment indicates that models achieve equivalent disambiguation despite qualitatively different layer-wise vulnerabilities. These findings offer a mechanism for recent observations where models' internal embedding similarities show low correlation with their behavioural outputs despite strong performance. The semantic distinctions therefore remain present but become increasingly invisible to similarity measures over the embeddings, with implications for embedding-based methods such as semantic search, retrieval, and clustering that rely on late-layer cosine similarity.
Sources
- Eliciting Latent Predictions from Transformers with the Tuned Lens
- The Llama 3 Herd of Models
- How to use and interpret activation patching
- The Remarkable Robustness of LLMs: Stages of Inference?
- Attention Sinks and Compression Valleys in LLMs are Two Sides of the Same Coin
- Evaluating Contextualized Representations of (Spanish) Ambiguous Words: A New Lexical Resource and Empirical Analysis
- Layer by Layer: Uncovering Hidden Representations in Language Models
- All Bark and No Bite: Rogue Dimensions in Transformer Language Models Obscure Representational Quality
- Qwen2.5 Technical Report
Related papers
- Exploring Solution Divergence and Its Effect on Large Language Model Problem Solving
- Ishigaki-IDS-Bench: A Benchmark for Generating Information Delivery Specification from BIM Information Requirements
- Subliminal Steering: Stronger Encoding of Hidden Signals
- MedStruct-S: A Benchmark for Key Discovery, Key-Conditioned QA and Semi-Structured Extraction from OCR Clinical Reports
- The End of Transformers? On Challenging Attention and the Rise of Sub-Quadratic Architectures
- Untangling the Mechanisms of Misleading Context in Medical Question Answering