Identifying and Mitigating Bottlenecks in Role-Playing Agents: A Systematic Study of Disentangling Character Profile Axes
cs.CL
Submitted: 2026-01-08
Updated: 2026-08-31
Comments: Accepted EMNLP 2026 Main
Code: https://github.com/kwaroran/Risuai
License: http://creativecommons.org/licenses/by/4.0/
The gist: While Large Language Model (LLM) role-playing agents have advanced rapidly, it remains unclear which profile elements genuinely drive role-playing quality.
Terminology
Abstract
While Large Language Model (LLM) role-playing agents have advanced rapidly, it remains unclear which profile elements genuinely drive role-playing quality. To bridge this gap, we introduce a systematic diagnostic framework that disentangles the impact of character profiles along three axes: Familiarity (Known vs. Unknown), Structure (Structured vs. Unstructured), and Disposition (Moral vs. Immoral). Utilizing a unified hierarchical schema (5 dimensions, 28 fields), we construct a controlled dataset of 211 personas and evaluate five LLMs on both single- and multi-turn interactions. Our results reveal a striking asymmetry: Familiarity and Structure show negligible impact, while Disposition produces large, consistent performance degradation for immoral characters across all conditions. Further analyses suggest that the Moral--Immoral gap is amplified by post-SFT alignment, and that this degradation varies substantially across profile attributes. To mitigate this bottleneck, we propose Field-Aware Contrastive Decoding (FACD), a training-free strategy that amplifies suppressed disposition-sensitive signals, significantly closing the performance gap without sacrificing moral-character performance.
Sources
- gpt-oss-120b & gpt-oss-20b Model Card
- Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback
- Examining Identity Drift in Conversations of LLM Agents
- Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilities
- Steer Model beyond Assistant: Controlling System Prompt Strength via Contrastive Decoding
- Gemini Embedding: Generalizable Embeddings from Gemini
- LLM Agents in Interaction: Measuring Personality Consistency and Linguistic Alignment in Interacting Populations of Large Language Models
- ChatHaruhi: Reviving Anime Character in Reality via Large Language Model
- Does Prompt Formatting Have Any Impact on LLM Performance?
- Safety Tax: Safety Alignment Makes Your Large Reasoning Models Less Reasonable
- DeepSeek-V3.2: Pushing the Frontier of Open Large Language Models
- GPT-4o System Card
- Ministral 3
- Olmo 3
- PersonaGym: Evaluating Persona Agents and LLMs
- Qwen3 Technical Report
- OpenCharacter: Training Customizable Role-Playing LLMs with Large-Scale Synthetic Personas
- A Survey of Large Language Models
Related papers
- Exploring Solution Divergence and Its Effect on Large Language Model Problem Solving
- Ishigaki-IDS-Bench: A Benchmark for Generating Information Delivery Specification from BIM Information Requirements
- Subliminal Steering: Stronger Encoding of Hidden Signals
- MedStruct-S: A Benchmark for Key Discovery, Key-Conditioned QA and Semi-Structured Extraction from OCR Clinical Reports
- The End of Transformers? On Challenging Attention and the Rise of Sub-Quadratic Architectures
- Untangling the Mechanisms of Misleading Context in Medical Question Answering