RMSWeb: Reflection, Failure-Mode Mining, and Salvage-DS for Web Agent Reinforcement Learning

arXiv:2608.00335 · cs.AI, cs.CL, cs.LG · Submitted 2026-07-31 · Read on arXiv

Chengbo Liu, Lifang Zhou, Ruijie Yan, Pei Tan, Ao Sun, Haojun Huang, Guichun Hua, Sining Wei, Yining Chen, Yingying He, Yutao Xie

cs.AI, cs.CL, cs.LG

Submitted: 2026-07-31

Comments: 15 pages, 9 figures, and 6 tables. Includes appendices

Code: https://github.com/browser-use/browser-use

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Terminology

• Benchmark — 16× in this paper · explained in 2 episodes
• Reinforcement Learning — 11× in this paper · explained in 4 episodes
• Localization — 10× in this paper · explained in 2 episodes
• Ablation — 9× in this paper · explained in 1 episode
• GRPO — 9× in this paper · explained in 6 episodes
• Harness — 9× in this paper · explained in 2 episodes
• Accuracy — 8× in this paper · explained in 1 episode
• Evaluation Protocol — 7× in this paper · explained in 2 episodes
• Grounding — 5× in this paper · explained in 3 episodes
• Perplexity — 5× in this paper · explained in 3 episodes
• Verifier — 5× in this paper · explained in 2 episodes
• Visual Grounding — 3× in this paper · explained in 2 episodes
• AI Agents — 2× in this paper · explained in 2 episodes
• Composition — 2× in this paper · explained in 2 episodes
• Core — 2× in this paper · explained in 2 episodes
• False Negatives — 2× in this paper · explained in 2 episodes
• Fine-Tuning — 2× in this paper · explained in 5 episodes
• Foundation Model — 2× in this paper · explained in 3 episodes
• LLM judge — 2× in this paper · explained in 2 episodes
• Papers — 2× in this paper · explained in 3 episodes
• Reinforcement Learning (RL) — 2× in this paper · explained in 26 episodes
• Supervised Fine-Tuning (SFT) — 2× in this paper · explained in 13 episodes
• Trace — 2× in this paper · explained in 2 episodes
• Ablation Studies — 1× in this paper · explained in 1 episode
• Actor — 1× in this paper · explained in 3 episodes
• Augmentation — 1× in this paper · explained in 2 episodes
• Benchmarking — 1× in this paper · explained in 4 episodes
• Distribution Shift — 1× in this paper · explained in 12 episodes
• Evaluation Framework — 1× in this paper · explained in 3 episodes
• Foundation Models — 1× in this paper · explained in 8 episodes
• Redundancy — 1× in this paper · explained in 2 episodes
• Steering — 1× in this paper · explained in 2 episodes
• Unsupervised Learning — 1× in this paper · explained in 2 episodes
• Verifiable Rewards — 1× in this paper · explained in 2 episodes
• Verification — 1× in this paper · explained in 2 episodes
• Vision-language models — 1× in this paper · explained in 2 episodes
• Weight Decay — 1× in this paper · explained in 2 episodes
• World Model — 1× in this paper · explained in 8 episodes

Sources

• Fara-7B: An Efficient Agentic Model for Computer Use
• The BrowserGym Ecosystem for Web Agent Research
• Is Your LLM Secretly a World Model of the Internet? Model-Based Planning for Web Agents
• MolmoWeb: Open Visual Web Agent and Open Data for the Open Web
• OpenWebVoyager: Building Multimodal Web Agents via Iterative Real-World Exploration, Feedback and Optimization
• Weblica: Scalable and Reproducible Training Environments for Visual Web Agents
• Tree Search for Language Model Agents
• GUI-R1 : A Generalist R1-Style Vision-Language Action Model For GUI Agents
• WebGPT: Browser-assisted question-answering with human feedback
• Agent Q: Advanced Reasoning and Learning for Autonomous AI Agents
• UI-TARS: Pioneering Automated GUI Interaction with Native Agents
• Proximal Policy Optimization Algorithms
• DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models
• Learn-by-interact: A Data-Centric Framework for Self-Adaptive Agents in Realistic Environments
• Agent Workflow Memory
• Mobile-Agent-v3.5: Multi-platform Fundamental GUI Agents
• M$^2$: Dual-Memory Augmentation for Long-Horizon Web Agents via Trajectory Summarization and Insight Retrieval
• Set-of-Mark Prompting Unleashes Extraordinary Visual Grounding in GPT-4V
• OpenWebRL: Demystifying Online Multi-turn Reinforcement Learning for Visual Web Agents
• DAPO: An Open-Source LLM Reinforcement Learning System at Scale

Related papers

• MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
• Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
• Agents' Last Exam
• The Clinician's Veto: Navigating Trust, Liability, and Uncertainty in Autonomous AI Prescribing
• MindHelper: Closed-Loop Embodied Mental-State Reasoning for Precision Intervention
• Incumbent Advantage: Brand Bias and Cognitive Manipulation Dynamics in LLM Recommendation Systems