"Act Like a 5th Grader" is Not Enough: Bounding Knowledge in LLM-Based User Simulators

arXiv:2608.30033 · cs.CL, cs.AI · Submitted 2026-08-30 · Read on arXiv

cs.CL, cs.AI

Submitted: 2026-08-30

Updated: 2026-08-30

Comments: Findings of the Association for Computational Linguistics: EMNLP 2026

Code: https://github.com/iai-group/emnlp2026-simreading

License: http://creativecommons.org/licenses/by/4.0/

The gist: Large language models (LLMs) are increasingly used to simulate human behavior but frequently fail to exhibit realistic cognitive constraints, suffering from a "superhuman bias." Using a dataset of

Terminology

Abstract

Large language models (LLMs) are increasingly used to simulate human behavior but frequently fail to exhibit realistic cognitive constraints, suffering from a "superhuman bias." Using a dataset of over 71,000 reading comprehension responses from 2,359 primary-school students (grades 4--6), we demonstrate that standard persona prompting yields near-perfect, deterministic performance, failing to capture the natural variance of developing readers. To address this, we introduce the Cognitively Bounded User Simulator (CBUS), an architectural framework that explicitly models the restricted working memory of young readers through an episodic bottleneck. Within this framework, we formalize two distinct test-taking strategies to emulate different reading behaviors. Our evaluation shows that explicitly modeling cognitive bounds significantly narrows the simulation gap across multiple LLM backbones, demonstrating that enforcing architectural constraints is more effective for high-fidelity simulation than simply scaling raw model capabilities.

Sources

Related papers