UserToolBench: A User-Profile-Hidden Benchmark for Personalized Decision Making in Tool-Use LLMs

UserToolBench: A User-Profile-Hidden Benchmark for Personalized Decision Making in Tool-Use LLMs

Summary

This paper introduces UserToolBench, a benchmark designed to evaluate personalized decision making in tool-use LLMs. The core problem addressed is that tool-use LLMs are increasingly expected to act on users' behalf, but existing benchmarks typically focus on profile recall, style imitation, generic tool use, or response-level personalization. UserToolBench instead tests whether a model can infer latent user preferences from interaction history, recognize when clarification is needed, and produce user-aligned tool-call trajectories under incomplete information.

The benchmark is constructed from privacy-sanitized real interaction traces and combines structured persona profiles, public API-style tool ecosystems, and long-horizon multi-turn trajectories. It includes 10 user profiles, 36 tool sets, 1,065 turns, and 170 unique tools. The evaluation focuses on three task types: lack-of-information (34.79%), single-tool (33.17%), and multi-tool (32.04%) settings. Turn-level subtypes include partial information (54.93%), cross-topic (28.17%), and long-range dependency (16.90%).

The central design feature is a profile-hidden evaluation protocol. Reference trajectories are constructed and validated with access to persistent user profiles, so target decisions are grounded in stable user preferences. During evaluation, the tested LLM does not observe the explicit profile; it receives only the accumulated interaction history, the current request, and the available tool schemas, and must infer the user-specific constraints needed for action. This asymmetry is formalized as: the reference trajectory is generated under y⋆ = f ref(p, h, q, T) where p is visible, while the evaluated LLM produces ŷ = f θ(h, q, T) where p is hidden.

The paper formulates personalized decision making as a sequential tool-use problem under incomplete information. The predicted trajectory may include clarification queries, tool invocations with arguments, interpretation of tool feedback, and final user-facing responses. Because q may omit decision-critical constraints, the LLM must choose whether to ask for clarification, infer missing constraints from history, or proceed with available information.

Experiments evaluate nine representative tool-use LLMs, including six strong general-purpose models (Kimi K2.6, GPT-5.4, Qwen 3.6 Plus, DeepSeek V4 Pro, Gemini 3.5 Flash, GLM-5) and three smaller tool-oriented models (Hammer2.1-7B, ToolACE-2.5-8B, Watt-Tool-8B). Key findings include:

1. Current models still struggle with strict personalized delegation. Even the best-performing model (GLM-5) reaches only 49.36% Avg. Exact Acc., showing that no tested model can reliably recover the profile-conditioned reference trajectory when the explicit user profile is hidden.

2. Executable task completion does not imply personalized decision alignment. There is a large gap between relaxed task completion and exact trajectory matching. For example, DeepSeek V4 Pro achieves 72.55% Avg. Relaxed Acc. but only 42.49% Avg. Exact Acc.; GPT-5.4 shows an even larger gap, from 31.98% exact accuracy to 67.71% relaxed accuracy.

3. Strict mismatches are frequently substantive rather than cosmetic. A multi-label analysis of failed trajectories finds that sequence/dependency errors occur in 82.1–96.5% of failures across the seven analyzed models, while wrong-tool decisions occur in 48.2–90.3% and user-constraint violations in 40.9–53.5%. A severity audit reports only 0.3–2.1 percentage points of slight preference deviation, compared with 28.8–65.5 percentage points of major decision deviation.

4. Multi-tool delegation remains a major bottleneck. Average accuracy drops from 51.30% on single-tool tasks to 25.24% on multi-tool tasks, reflecting a sequential decision-control problem under personalization constraints.

5. Preference inference under missing information is not captured by generic tool-use ability. For instance, GPT-5.4 obtains the highest LOI accuracy at 42.22% despite its lower overall exact accuracy, whereas Qwen 3.6 Plus performs strongly on single-tool tasks at 58.02% but drops to 19.52% on LOI tasks.

6. Accumulated interaction history does not uniformly improve personalized delegation. Several models degrade in later trajectory stages; for example, GPT-5.4 drops from 52.12% in the first third to 31.33% in the last third, and Qwen 3.6 Plus declines from 45.93% to 28.56%. However, Gemini 3.5 Flash improves in later stages.

The paper also includes a qualitative case study illustrating the distinction between surface-level task completion and profile-conditioned decision alignment. In this case, a GPT model directly calls a metadata tool with a guessed URL instead of first grounding the missing URL through search, demonstrating a breakdown in history-dependent preference inference, uncertainty handling, and grounded tool-call planning.

The paper's contributions are threefold: (1) formulating personalized decision making as an incomplete-information tool-use problem grounded in persistent user behavior, (2) proposing a profile-hidden evaluation protocol that tests whether LLMs can infer latent user preferences from interaction history and express them through executable tool-call trajectories, and (3) evaluating strong tool-use LLMs and showing that current models still struggle with reliable personalized delegation.

The paper concludes that personalization evaluation should move beyond asking whether outputs sound user-specific and instead ask whether LLMs make correct decisions for the users they represent. Code and resources are available at https://github.com/xxy212/UserToolBench.

Improvements to AI systems:

1. Add a preference-inference module that explicitly models latent user constraints from interaction history. The system should maintain a dynamic user-state graph (e.g., preferred tools, argument formats, domain-specific shortcuts) updated after each turn, and use it to condition tool-call decisions even when the current request omits critical details. This directly addresses the 34.79% lack-of-information tasks where models fail to infer missing constraints.

2. Implement a clarification-or-inference decision gate. Before invoking any tool, the system should compute a confidence score for each required argument based on history-derived evidence. If confidence is below a threshold (e.g., 0.6), it must generate a clarification query instead of guessing. This reduces the 40.9–53.5% user-constraint violations and prevents the observed failure mode where models guess URLs instead of searching.

3. Add a multi-tool sequential planner with dependency tracking. For multi-tool tasks (32.04% of the benchmark), the system should explicitly represent inter-tool dependencies (e.g., output of tool A is input to tool B) and verify that each step’s output satisfies the next step’s schema constraints before proceeding. This targets the 82.1–96.5% sequence/dependency errors and the accuracy drop from 51.30% to 25.24% on multi-tool tasks.

4. Introduce a history-relevance weighting mechanism. The system should assign decaying or topic-based weights to past interactions, prioritizing recent and topically similar turns when inferring preferences. This mitigates the observed degradation in later trajectory stages (e.g., GPT-5.4 dropping from 52.12% to 31.33% accuracy) by preventing irrelevant or outdated history from overriding current user intent.

5. Add a post-hoc trajectory self-check for alignment. After generating a tool-call sequence, the system should run a lightweight validator that checks: (a) all user-visible constraints from history are satisfied, (b) no tool is called with unverified arguments, and (c) the final response explicitly states any assumptions made. This converts the gap between relaxed and exact accuracy (e.g., 72.55% vs. 42.49%) into a self-correctable signal.

6. Train a separate preference-encoder for profile-hidden settings. Instead of relying solely on the base LLM’s generic tool-use ability, the system should include a small, specialized encoder that takes the interaction history and outputs a compact preference vector (e.g., 128 dimensions), which is then concatenated with the current request and tool schemas for decision-making. This decouples preference inference from tool execution, addressing the finding that high generic tool-use ability (e.g., GPT-5.4’s 42.22% LOI accuracy) does not correlate with overall personalized delegation.

What the improved AI system can do:

• Correctly infer missing user-specific arguments (e.g., preferred travel class, budget range, default file format) from history without explicit profile access, reducing clarification requests to only genuinely ambiguous cases.

• Execute multi-tool workflows (e.g., book flight → reserve hotel → schedule calendar) with correct argument chaining, achieving near-single-tool accuracy on multi-tool tasks.

• Maintain consistent decision quality across long conversations, avoiding late-stage degradation by weighting recent and relevant history more heavily.

• Detect and flag its own potential misalignments (e.g., “I assumed your usual 3-star hotel preference”) before finalizing actions, allowing users to correct course early.

• Distinguish between surface-level task completion (e.g., returning a search result) and user-aligned action (e.g., booking the specific hotel the user would choose), matching reference trajectories in exact tool calls and arguments.