When Users Change Their Minds: Measuring and Repairing Intent Drift in LLM Agents
cs.CL, cs.AI
Submitted: 2026-09-26
Updated: 2026-09-26
Comments: 15 pages, 3 figures
License: http://creativecommons.org/licenses/by/4.0/
The gist: LLM agents often operate over multi-turn interactions in which user intent changes before execution.
Terminology
Abstract
LLM agents often operate over multi-turn interactions in which user intent changes before execution. We study intent drift: the failure mode in which superseded parts of the user's intent continue to influence the final answer or tool action. We introduce IntentFlux, an executable benchmark that converts verifiable tasks into dialogues with controlled intent changes while preserving their original graders. In a 627-case calibration, mean task score falls from 0.476 to 0.384 as dialogues contain more superseded and withdrawn information. Across eight models, the rate of fully correct solutions is significantly lower when the same final task must be recovered from an evolving dialogue rather than given directly in a single turn. We further introduce StateForge, which explicitly maintains the active requirements before generation. On General-Test, it improves mean task score from 0.367 to 0.467. Providing the ground-truth final state improves performance further but still does not recover single-turn performance, indicating that state-estimation errors explain only part of the gap. These results establish intent drift as a measurable multi-turn failure mode and explicit state maintenance as a partial mitigation.
Sources
- Evaluating Large Language Models Trained on Code
- Deliberative Alignment: Reasoning Enables Safer Language Models
- Orca: Progressive Learning from Complex Explanation Traces of GPT-4
- Supersede: Diagnosing and Training the Memory-Update Gap in LLM Agents
- LLMs Get Lost in Evolving User Intent
- When Users Change Their Mind: Evaluating Interruptible Agents in Long-Horizon Web Navigation
Related papers
- Exploring Solution Divergence and Its Effect on Large Language Model Problem Solving
- Ishigaki-IDS-Bench: A Benchmark for Generating Information Delivery Specification from BIM Information Requirements
- Subliminal Steering: Stronger Encoding of Hidden Signals
- MedStruct-S: A Benchmark for Key Discovery, Key-Conditioned QA and Semi-Structured Extraction from OCR Clinical Reports
- The End of Transformers? On Challenging Attention and the Rise of Sub-Quadratic Architectures
- Untangling the Mechanisms of Misleading Context in Medical Question Answering