Training Computer Use Agents to Assess the Usability of Graphical User Interfaces
cs.CL, cs.AI
Submitted: 2026-04-28
Updated: 2026-11-10
Terminology
Sources
- Back to Basics: Revisiting REINFORCE Style Optimization for Learning from Human Feedback in LLMs
- Multimodal Web Navigation with Instruction-Finetuned Foundation Models
- GUITester: Enabling GUI Agents for Exploratory Defect Discovery
- A Real-World WebAgent with Planning, Long Context Understanding, and Program Synthesis
- UXCascade: Scalable Usability Testing with Simulated User Agents
- WebDevJudge: Evaluating (M)LLMs as Critiques for Web Development Quality
- Let's Verify Step by Step
- Social Simulacra: Creating Populated Prototypes for Social Computing Systems
- DesignPref: Capturing Personal Preferences in Visual Design Generation
- UI-TARS: Pioneering Automated GUI Interaction with Native Agents
- DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models
- Rethinking Rubric Generation for Improving LLM Judge and Reward Modeling for Open-ended Tasks
- JudgeBench: A Benchmark for Evaluating LLM-based Judges
- Kimi K2.5: Visual Agentic Intelligence
- Llama 2: Open Foundation and Fine-Tuned Chat Models
- AndroidEnv: A Reinforcement Learning Platform for Android
- OpenCUA: Open Foundations for Computer-Use Agents
- UIClip: A Data-driven Model for Assessing User Interface Design
- Improving User Interface Generation Models from Designer Feedback
- EvoCUA: Evolving Computer Use Agents via Learning from Scalable Synthetic Experience
Related papers
- ALARM: Audio-Language Alignment for Reasoning Models
- ThinkSum: Probabilistic reasoning over sets using large language models
- Coherence boosting: When your pretrained language model is not paying enough attention
- From Facts to Insights: A Persona-Driven Dual Memory Framework and Dataset for Role-Playing Agents
- Improving General Role-Playing Agents via Psychology-Grounded Reasoning and Role-Aware Policy Optimization
- The Art of Asking: Multilingual Prompt Optimization for Synthetic Data