Cultural Divergence Preservation: Diagnosing Flattening and Caricature in LLM-Simulated Survey Populations
cs.CL, cs.AI
Submitted: 2026-09-24
Updated: 2026-09-24
Comments: Accepted to the EMNLP 2026 Workshop on Pluralistic AI & NLP (PANDORA)
License: http://creativecommons.org/licenses/by/4.0/
The gist: Large language models (LLMs) are increasingly used as synthetic survey respondents to estimate population response distributions.
Terminology
Abstract
Large language models (LLMs) are increasingly used as synthetic survey respondents to estimate population response distributions. In cross-cultural survey simulation, evaluations should assess not only distributional fidelity within countries but also whether differences across countries are preserved. However, existing distance-based metrics such as Jensen--Shannon divergence (JSD) do not directly capture such cross-country differences. To address this limitation, we introduce Cultural Divergence Preservation (CDP), a reference-light diagnostic based on a one-time human calibration. CDP identifies reduced cross-country divergence as cultural flattening and increased divergence as cultural caricature. To evaluate CDP, we conduct experiments across four LLM backbones, three persona-based prompting methods, and two survey domains, the World Values Survey (WVS) and the Big Five Personality Test. The results reveal a systematic discrepancy between conventional fidelity metrics and CDP. Controlled experiments show that CDP changes monotonically as cross-country divergence is attenuated or amplified, while the corresponding changes in JSD remain relatively small. In our audit of real LLM generations, DeepPersona-Inspired prompting is frequently favored by conventional fidelity metrics but exhibits the strongest flattening in every model--domain block. CDP thus complements fidelity metrics by directly quantifying the attenuation or amplification of cross-country divergence.
Sources
- PLURAL: A Global Dataset for Value Alignment
- Whose Personae? Synthetic Persona Experiments in LLM Research and Pathways to Transparency
- Beyond the Mean: Three-Axis Fidelity for Aligning LLM-Based Survey Simulators from Small Pilot Data
- Polypersona: Persona-Grounded LLM for Synthetic Survey Responses
- Towards Measuring the Representation of Subjective Global Opinions in Language Models
- Scaling Synthetic Data Creation with 1,000,000,000 Personas
- Gemma 3 Technical Report
- This human study did not involve human subjects: Validating LLM simulations as behavioral evidence
- The American Ghost in the Machine: How language models align culturally and the effects of cultural prompting
- Assessing the Reliability of Persona-Conditioned LLMs as Synthetic Survey Respondents
- Qwen3.5-Omni Technical Report
- Llama 2: Open Foundation and Fine-Tuned Chat Models
- OpenCharacter: Training Customizable Role-Playing LLMs with Large-Scale Synthetic Personas
- DeepPersona: A Generative Engine for Scaling Deep Synthetic Personas
- A Survey on Evaluating Quality and Trustworthiness in LLM-Generated Data
Related papers
- Exploring Solution Divergence and Its Effect on Large Language Model Problem Solving
- Ishigaki-IDS-Bench: A Benchmark for Generating Information Delivery Specification from BIM Information Requirements
- Subliminal Steering: Stronger Encoding of Hidden Signals
- MedStruct-S: A Benchmark for Key Discovery, Key-Conditioned QA and Semi-Structured Extraction from OCR Clinical Reports
- The End of Transformers? On Challenging Attention and the Rise of Sub-Quadratic Architectures
- Untangling the Mechanisms of Misleading Context in Medical Question Answering