Beyond Borrowed Histories: Person-Aligned User Simulation for Interactive Role-Playing Evaluation
Yuhang Zhu, Mingxuan Du, Benfeng Xu, Jie Gao, Lingyun Yu, Hongtao Xie
cs.CL, cs.AI
Submitted: 2026-07-30
Comments: 29 pages, 3 figures, including supplementary material. Resources: https://github.com/Zhuyh1139/PALATE
Code: https://github.com/Zhuyh1139/PALATE
License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
Terminology
Sources
- PingPong: A Benchmark for Role-Playing Language Models with User Emulation and Multi-Model Evaluation
- Rewarding Chatbots for Real-World Engagement with Millions of Users
- RoleEval: A Bilingual Role Evaluation Benchmark for Large Language Models
- OpenCharacter: Training Customizable Role-Playing LLMs with Large-Scale Synthetic Personas
- Learning User Simulators with Turing Rewards
- $\tau$-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domains
Related papers
- Exploring Solution Divergence and Its Effect on Large Language Model Problem Solving
- Ishigaki-IDS-Bench: A Benchmark for Generating Information Delivery Specification from BIM Information Requirements
- Subliminal Steering: Stronger Encoding of Hidden Signals
- MedStruct-S: A Benchmark for Key Discovery, Key-Conditioned QA and Semi-Structured Extraction from OCR Clinical Reports
- The End of Transformers? On Challenging Attention and the Rise of Sub-Quadratic Architectures
- Untangling the Mechanisms of Misleading Context in Medical Question Answering