When Synthetic Users Fail: A Cross-Domain Benchmark of LLM-Simulated Human Survey Responses
This study tested if LLM synthetic users are trustworthy for decision support by simulating human survey responses across two domains: U.S. social attitudes and cross-cultural values. Findings show two consistent failures: LLMs lack individual fidelity compared to baselines, and they exhibit demographic over-determination, exaggerating how strongly demographics predict answers.