Cultural Awareness is Represented but Not Decoded: Tracing Mythological Knowledge across 18 Open-Source LLMs
Iaroslav Chelombitko, Ekaterina Chelombitko, Mika Hämäläinen
cs.CL, cs.CY, cs.LG
Submitted: 2026-08-03
Comments: 45 pages, 23 figures, 18 tables. Dataset: https://huggingface.co/datasets/Aragoner/folkmotif Code: https://github.com/AragonerUA/folkmotif
Code: https://github.com/AragonerUA/folkmotif
License: http://creativecommons.org/licenses/by/4.0/
The gist: Open-source LLMs reliably name Zeus, Jupiter, and Thor, but recover their counterparts in less-represented traditions like Finnish, Slavic, Egyptian, or Chinese mythology far less consistently.
Terminology
Abstract
Open-source LLMs reliably name Zeus, Jupiter, and Thor, but recover their counterparts in less-represented traditions like Finnish, Slavic, Egyptian, or Chinese mythology far less consistently. We ask where inside the model this cultural default is produced. On a parallel cross-cultural substrate of Thompson-motif entities, we instrument 18 open-source LLMs from 8 architecture families with linear probing, logit lens, activation patching, and output extraction. The residual stream cleanly distinguishes cultures, well above a name-string baseline, yet the decoder collapses culturally-specific tokens onto dominant-tradition ones. The failure is at readout, not at representation. Asking the same question in the target culture's native language versus English produces failures that cluster within language but decouple across language: the decoder is gated on prompt language. We release a per-entity (probe, output) decomposition framework, a citation-anchored cross-cultural ground truth, a within- versus cross-mode correlation test for language-conditioned readout, and per-entity predictions for all 18 models.
Sources
- Phi-4 Technical Report
- Towards Measuring and Modeling "Culture" in LLMs: A Survey
- Do All Languages Cost the Same? Tokenization in the Era of Commercial Language Models
- Yi: Open Foundation Models by 01.AI
- Understanding intermediate layers using linear classifier probes
- Aya 23: Open Weight Releases to Further Multilingual Progress
- Eliciting Latent Predictions from Transformers with the Tuned Lens
- Subword-Based Comparative Linguistics across 242 Languages Using Wikipedia Glottosets
- Qtok: A Comprehensive Framework for Evaluating Multilingual Tokenizer Quality in Large Language Models
- LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scale
- GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers
- Datasheets for Datasets
- Finding Alignments Between Interpretable Causal Variables and Distributed Neural Representations
- The Llama 3 Herd of Models
- Language Models Represent Space and Time
- How Does Quantization Affect Multilingual LLMs?
- The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasets
- Locating and Editing Factual Associations in GPT
- 2 OLMo 2 Furious
- Language Models as Knowledge Bases?
Related papers
- Exploring Solution Divergence and Its Effect on Large Language Model Problem Solving
- Ishigaki-IDS-Bench: A Benchmark for Generating Information Delivery Specification from BIM Information Requirements
- Subliminal Steering: Stronger Encoding of Hidden Signals
- MedStruct-S: A Benchmark for Key Discovery, Key-Conditioned QA and Semi-Structured Extraction from OCR Clinical Reports
- The End of Transformers? On Challenging Attention and the Rise of Sub-Quadratic Architectures
- Untangling the Mechanisms of Misleading Context in Medical Question Answering