LLMs Infer Cultural Context but Fail to Apply It When Responding
cs.CL
Submitted: 2026-06-16
Updated: 2026-10-15
Comments: Accepted to EMNLP 2026 (Main Conference); 10 pages, 7 figures, 2 tables (26 pages, 13 figures, 8 tables including references and appendices)
Code: https://github.com/YisongMiao/CAPRI
License: http://creativecommons.org/licenses/by/4.0/
The gist: Recent work has shown that LLMs overrepresent dominant cultures, particularly Western ones, while marginalizing others.
Terminology
Abstract
Recent work has shown that LLMs overrepresent dominant cultures, particularly Western ones, while marginalizing others. We investigate whether this affects models' ability to generate culturally adapted responses by evaluating their use of local measurement units based on the user's perceived cultural background. We introduce Cultural and Pragmatic Response Inference (CAPRI), a dataset of conversations with varying levels of cultural cues. Experiments with state-of-the-art LLMs show that models can infer cultural background and recall relevant conventions, but often fail to utilize the information to adapt their answers to the relevant cultural conventions, unless explicitly prompted to perform the tasks sequentially. We further evaluate adaptation to the interpretation of time and quantity expressions, two subjective language grounding dimensions that are affected by culture. We find that models increasingly adapt their answers as cultural cues accumulate, but their priors are not culture-neutral, sometimes aligning with the model's country of origin. Overall, CAPRI provides a resource for future research aimed at narrowing the gap between cultural knowledge and culturally adaptive language generation.
Sources
- Qwen3-VL Technical Report
- Pragmatic Reasoning improves LLM Code Generation
- The Llama 3 Herd of Models
- Are LLMs good pragmatic speakers?
- Is It Bad to Work All the Time? Cross-Cultural Evaluation of Social Norm Biases in GPT-4
- Location Not Found: Exposing Implicit Local and Global Biases in Multilingual LLMs
- How Hypocritical Is Your LLM judge? Listener-Speaker Asymmetries in the Pragmatic Competence of Large Language Models
Related papers
- Exploring Solution Divergence and Its Effect on Large Language Model Problem Solving
- Ishigaki-IDS-Bench: A Benchmark for Generating Information Delivery Specification from BIM Information Requirements
- Subliminal Steering: Stronger Encoding of Hidden Signals
- MedStruct-S: A Benchmark for Key Discovery, Key-Conditioned QA and Semi-Structured Extraction from OCR Clinical Reports
- The End of Transformers? On Challenging Attention and the Rise of Sub-Quadratic Architectures
- Untangling the Mechanisms of Misleading Context in Medical Question Answering