DIAL: Direct Iterative Adversarial Learning for Realistic Multi-Turn Dialogue Simulation
cs.CL
Submitted: 2025-12-23
Updated: 2026-11-03
Journal ref: Findings of the Association for Computational Linguistics: EMNLP 2026, pages 925-942, Budapest, Hungary, October 2026
DOI: 10.18653/2026.findings-emnlp.46
Code: https://github.com/slingshot-ai/dial
License: http://creativecommons.org/licenses/by/4.0/
The gist: Realistic user simulation is crucial for training and evaluating multi-turn dialogue systems, yet creating simulators that accurately replicate human behavior remains a significant challenge.
Terminology
Abstract
Realistic user simulation is crucial for training and evaluating multi-turn dialogue systems, yet creating simulators that accurately replicate human behavior remains a significant challenge. An effective simulator must expose the failure modes of the systems under evaluation. This work introduces Direct Iterative Adversarial Learning (DIAL), an adversarial framework that iteratively enhances user simulator realism through a competitive dynamic between a generator (user simulator) and a discriminator. When applied to mental health support, a domain characterized by diverse failure types and a critical dependence on realistic user behavior for failure detection, DIAL restores lexical diversity diminished by supervised fine-tuning and drastically reduces discriminator accuracy. The resulting simulator exhibits a strong correlation between simulated and real failure occurrence rates while maintaining low distributional divergence of failure modes. These findings indicate that DIAL is a promising method for developing realistic user simulators in multi-turn dialogue, facilitating reliable and cost-effective system evaluation prior to deployment.
Sources
- Kimi K2: Open Agentic Intelligence
- The Llama 3 Herd of Models
- Zero-Shot Goal-Directed Dialogue via RL on Imagined Conversations
- Interactive Dialogue Agents via Reinforcement Learning on Hindsight Regenerations
- GPT-4o System Card
- Adversarial learning of neural user simulators for dialogue policy optimisation
- Efficient RL for optimizing conversation level outcomes with an LLM-based tutor
- MindEval: Benchmarking Language Models on Multi-turn Mental Health Support
- Proximal Policy Optimization Algorithms
- Multi-turn Reinforcement Learning from Preference Human Feedback
- DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models
- OpenAI GPT-5 System Card
- Qwen2.5 Technical Report
- $\tau$-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domains
- Reflective Preference Optimization (RPO): Enhancing On-Policy Alignment via Hint-Guided Reflection
- CyclicJudge: Mitigating Judge Bias Efficiently in LLM-based Evaluation
Related papers
- ALARM: Audio-Language Alignment for Reasoning Models
- ThinkSum: Probabilistic reasoning over sets using large language models
- Coherence boosting: When your pretrained language model is not paying enough attention
- From Facts to Insights: A Persona-Driven Dual Memory Framework and Dataset for Role-Playing Agents
- Improving General Role-Playing Agents via Psychology-Grounded Reasoning and Role-Aware Policy Optimization
- The Art of Asking: Multilingual Prompt Optimization for Synthetic Data