Can Agents Work for Everyone? Cross-User Reliability for Mobile GUI Agents in Personalized User Interfaces
cs.AI
Submitted: 2026-10-06
Updated: 2026-10-06
Comments: 23 pages, 12 figures
License: http://creativecommons.org/licenses/by/4.0/
The gist: Mobile GUI agents increasingly operate on interfaces influenced by users' histories and preferences, but their reliability across different users remains underexplored.
Abstract
Mobile GUI agents increasingly operate on interfaces influenced by users' histories and preferences, but their reliability across different users remains underexplored. We introduce PAIR (Personalized Application-state Instantiation and Rendering), a pipeline for constructing user-conditioned application states that enables controlled evaluation of the same task across different users. We further introduce RePAIR (Reinforcement learning with Personalization-Aware Interaction Rewards), a training approach that learns from cross-user differences in subgoal outcomes to improve reliability across user-conditioned mobile environments. Across six agents, we find substantial variation in task success across users and consistently lower subgoal achievement in user-conditioned UI contexts (6.98 to 15.4 pp). This gap further increases for personal targets drawn from each user's own content (8.77 to 22.0 pp). Failures in these contexts frequently involve selecting another item instead of the intended target, particularly before target exposure. Finally, RePAIR improves user-conditioned SAR (+5.87 pp), all-success (+7.50 pp), and overall Task SR (+9.42 pp) over its supervised fine-tuning parent on unseen users, providing initial evidence that explicitly learning from cross-user variation can improve GUI-agent reliability.
Sources
- Qwen3-VL Technical Report
- KnowU-Bench: Towards Interactive, Proactive, and Personalized Mobile Agent Evaluation
- LoRA: Low-Rank Adaptation of Large Language Models
- iOSWorld: A Benchmark for Personally Intelligent Phone Agents
- AutoGLM: Autonomous Foundation Agents for GUIs
- PSPA-Bench: A Personalized Benchmark for Smartphone GUI Agent
- gpt-oss-120b & gpt-oss-20b Model Card
- UI-TARS: Pioneering Automated GUI Interaction with Native Agents
- DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models
- Me-Agent: A Personalized Mobile Agent with Two-Level User Habit Learning for Enhanced Interaction
- GUI-Perturbed: Domain Randomization Reveals Systematic Brittleness in GUI Grounding Models
- MobileGym: A Verifiable and Highly Parallel Simulation Platform for Mobile GUI Agent Research
- Mobile-Agent-v3.5: Multi-platform Fundamental GUI Agents
- AppAgent-Pro: A Proactive GUI Agent System for Multidomain Information Integration and User Assistance
- MAI-UI Technical Report: Real-World Centric Foundation GUI Agents
Related papers
- MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
- Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
- Agents' Last Exam
- The Clinician's Veto: Navigating Trust, Liability, and Uncertainty in Autonomous AI Prescribing
- MindHelper: Closed-Loop Embodied Mental-State Reasoning for Precision Intervention
- Incumbent Advantage: Brand Bias and Cognitive Manipulation Dynamics in LLM Recommendation Systems