Intent Speaks Louder: Controllable User Simulation Beyond Response Imitation

Summary

The paper Intent Speaks Louder: Controllable User Simulation Beyond Response Imitation introduces UserIDA (User Intent-Directive Alignment), a framework for controllable user simulation that separates which local interaction intent the next user turn should realize from how that intent is expressed in language.

Problem and Key Insight

The authors argue that generating the next user turn is inherently one-to-many: the same profile and dialogue context may support multiple plausible continuations with different local interaction intents. A fluent response may therefore advance the dialogue through an inappropriate intent, such as acceptance rather than repair. The key insight is that controllable user simulation should separate which local interaction intent the next user turn should realize from how that intent is expressed in language. Existing user simulators typically infer the next local user action jointly with its surface realization rather than exposing it as an explicit per-turn control variable.

UserIDA Framework

UserIDA exposes interaction intent as an explicit per-turn directive and consists of three stages:

1. Canonical Intent Annotation: Defines a six-way intent interface with the following directives: Initiate (start a new top-level goal or thread), Amend (refine or update an ongoing task), Supply (provide requested information or a choice), Repair (correct or challenge an assistant mismatch), SetRegister (set the role, protocol, tone, or response mode), and GroundAccept (accept, acknowledge, continue, or close). These labels are turn-local, operationally distinguishable, and sufficiently abstract that they do not specify the target payload or wording.

2. Intent Supervised Fine-Tuning (Intent-SFT): Trains the simulator to generate the observed user turn from dialogue context, optional implicit user information, and the target directive, enabling different valid actions to be realized within the same conversation thread. The training objective is standard next-token prediction conditioned on the directive.

3. Intent-Calibrated Policy Optimization: Addresses the quality–intent mismatch where a semantically attractive but wrong-intent candidate may otherwise receive a favorable relative advantage. The method computes a group-specific calibration magnitude Δt = max(0, max(qi for violating) − min(qj for compliant) + m), where m is a fixed margin. Calibrated rewards are set to qi for compliant candidates and qi − Δt for violating candidates, guaranteeing maxi:bi=0 ri + m ≤ minj:bj=1 rj for every mixed group. This ensures no high-quality intent-violating candidate can outrank an intent-compliant alternative after calibration. The policy update follows Group Relative Policy Optimization (GRPO).

Experimental Setup

• Training Data: LMSYS-Chat-1M, preprocessed following USP protocol, yielding 444,635 non-ambiguous training turns, 27,778 validation turns, and 9,233 test turns.

• Annotation: A frozen Qwen3.5-9B retrospective verifier assigns canonical intents.

• Baselines: General LMs (LLaMA-3-8B Base, GPT-4o, Gemini-2.5-Flash) and dedicated simulators USP-8B and UserLM-8B.

• Metrics: Intent Accuracy, Macro-F1, SimCSE, StyleCSE, Contextual Validity, and User Authenticity.

Main Results

Turn-Level Results: UserIDA achieves 86.62% intent accuracy and 0.864 macro-F1, outperforming the strongest dedicated user-simulator baseline by 24.3 percentage points while improving semantic and stylistic similarity. Specifically, it exceeds USP w/ Directive (62.28%) by 24.34 points and the strongest general-model baseline (GPT-4o w/ Directive at 77.34%) by 9.28 points. The gains extend to minority intents—for Supply (only 1.9% of labeled turns), accuracy increases from 32.8% for USP to 73.4% for UserIDA.

Ablation Analysis: Adding Intent-SFT raises intent accuracy from 56.2% to 82.0% and SimCSE from 0.28 to 0.57. Quality-only GRPO (without intent calibration) raises SimCSE to 0.61 but lowers intent accuracy to 80.7%, directly demonstrating the quality-intent mismatch. Full UserIDA reaches 86.6% intent accuracy and 0.59 SimCSE.

Controlled Multi-Turn Trajectory Evaluation: UserIDA improves step intent accuracy by 28.21 points over USP w/ Directive (88.38% vs. 60.17%) and all-turn success by 45 points (58% vs. 13%). UserLM achieves stronger role-based rubric scores than USP but realizes all requested intents in only 2% of trajectories, illustrating that role plausibility and local control are distinct.

Within-Context Directive Intervention: In dialogue states where all six directives are judged feasible, UserIDA achieves 75.35% intent accuracy (vs. 22.92% for USP w/ Directive), increases 4-of-6 Success from 22.92% to 91.67%, and raises realized-intent coverage from 1.83 to 4.71. The paper notes the gain therefore reflects stronger control over the local interaction transition rather than a reduction to rigid or unnatural templates.

Context-Dependence Probe: The six intents exhibit distinct dependencies on dialogue state components. Amend is most sensitive to earlier history (ΔNLL = 0.786), Repair depends on both preceding assistant response (0.403) and earlier history (0.494), GroundAccept is especially sensitive to the preceding assistant response (0.651), and Initiate and SetRegister show stronger profile dependence (0.590 and 0.361).

Human Alignment: The retrospective verifier reaches 87.88% accuracy, 0.875 macro-F1, and κ = 0.8223 agreement with expert labels. The prospective predictor (without observing the next user turn) reaches 71.39% accuracy and 0.718 macro-F1, consistent with the one-to-many nature of next-user behavior.

Conclusion

The paper establishes local interaction intent as a complementary control dimension beyond response imitation. The quality-only GRPO ablation shows response similarity can improve while intent control degrades, motivating explicit calibration between compliant and violating candidates. The results demonstrate that per-turn intent control as a complementary dimension to response fidelity in user simulation.

Improvements to AI Systems:

1. Explicit Per-Turn Intent Control in Dialogue Agents

• The AI system can now be trained with a six-way intent directive (Initiate, Amend, Supply, Repair, SetRegister, GroundAccept) as an explicit control variable for each generated turn.

• This allows the system to separate what interaction goal a turn should achieve from how it is phrased, enabling users or downstream controllers to specify the intended local action (e.g., repair the assistant's misunderstanding vs. accept and continue) independently of surface style.

• The system can generate multiple valid continuations for the same context, each realizing a different intent, and can be steered per-turn without retraining.

2. Intent-Calibrated Reinforcement Learning for Policy Optimization

• The AI system can use a calibration mechanism that prevents high-quality but intent-violating outputs from outranking intent-compliant ones during policy updates.

• This guarantees that, for any mixed group of candidate responses, the best compliant response always has a higher reward than the best violating response, eliminating the quality–intent mismatch seen in standard RLHF/GRPO.

• The system can now optimize for both response fidelity and intent adherence simultaneously, without sacrificing one for the other.

3. Retrospective and Prospective Intent Verification

• The system can incorporate a frozen verifier model that retrospectively labels the intent of any generated turn with 87.88% accuracy and high inter-annotator agreement (κ = 0.82).

• A prospective predictor (trained without seeing the next turn) can estimate the distribution of feasible intents in a given dialogue state, enabling the system to anticipate which intents are contextually plausible and to flag low-probability or ambiguous transitions.

• This allows the AI to self-monitor its own intent adherence during generation and to provide explainable feedback on why a turn was chosen (e.g., this turn amends the task vs. this turn supplies missing info).

4. Context-Dependent Intent Sensitivity Modeling

• The system can learn which parts of the dialogue state (e.g., earlier history, preceding assistant response, user profile) most strongly influence each intent type.

• For example, it can learn that Repair depends on both the assistant's last response and earlier history, while GroundAccept is most sensitive to the immediate assistant turn.

• This enables the AI to allocate attention dynamically: when generating a Repair turn, it can weight the assistant's last utterance and long-term context more heavily; when generating a GroundAccept, it can focus on the immediate response. This improves both accuracy and efficiency in long dialogues.

5. Controllable Multi-Turn Trajectory Generation

• The system can now execute a sequence of user intents across multiple turns with high fidelity (88.38% step-level intent accuracy, 58% all-turn success), enabling realistic and controllable user simulation for training and evaluating assistant systems.

• It can also handle within-context directive interventions: given a dialogue state where multiple intents are feasible, the system can switch between them on command (e.g., from supply to repair mid-conversation) with 91.67% success on 4-of-6 intent sets, without producing unnatural or template-like outputs.

6. Minority Intent Handling

• The system can generate rare intents (e.g., Supply, only 1.9% of training data) with high accuracy (73.4% vs. 32.8% for prior baselines), making it robust to imbalanced intent distributions.

• This is achieved via the intent-conditioned training and calibration, which prevents the model from collapsing toward frequent intents (like GroundAccept) and ensures diverse, intent-appropriate behavior even for underrepresented actions.

7. Improved User Simulation for Assistant Evaluation

• The improved AI can serve as a user simulator that is both locally controllable (per-turn intent) and globally coherent (multi-turn goal pursuit).

• It can generate user turns that are semantically and stylistically similar to real users (SimCSE 0.59, StyleCSE improved) while also following explicit intent directives, enabling more rigorous and targeted stress-testing of assistant systems (e.g., testing how an assistant handles a user who repeatedly repairs or amends).

• This reduces the need for human role-play in dialogue evaluation and allows automated, reproducible testing of assistant robustness across diverse user behaviors.

8. Intent-Aware Data Annotation and Augmentation

• The system can automatically annotate large dialogue corpora with canonical intent labels using the retrospective verifier, enabling the creation of intent-labeled datasets for training other models.

• It can also generate synthetic dialogue continuations with specified intents for data augmentation, improving the diversity and coverage of training data for downstream tasks like dialogue policy learning or assistant response generation.

User simulators are widely used as scalable environments for training and evaluating interactive assistants. Generating the next user turn is inherently one-to-many: the same profile and dialogue context may support multiple plausible continuations with different local interaction intents. A fluent response may therefore advance the dialogue through an inappropriate intent, such as acceptance rather than repair. Our key insight is that controllable user simulation should separate which local interaction intent the next user turn should realize from how that intent is expressed in language. We introduce UserIDA (User Intent-Directive Alignment), which exposes interaction intent as an explicit per-turn directive. UserIDA defines a six-way intent interface, learns directive-conditioned generation through supervised fine-tuning, and uses intent-calibrated policy optimization during group-based reinforcement learning. The reward preserves composite response quality while ensuring that intent-violating candidates rank below compliant alternatives in mixed groups. On LMSYS-USP, UserIDA achieves 86.6% intent accuracy, outperforming the strongest dedicated user-simulator baseline by 24.3 percentage points while improving semantic and stylistic similarity. In within-context interventions, it realizes at least four of the six target intents in 91.7% of evaluated dialogue states, compared with 22.9% for the strongest external baseline. These results establish per-turn intent control as a complementary dimension to response fidelity in user simulation.