RoboARCade: Benchmarking Agentic Robot Control
cs.RO
Submitted: 2026-11-05
Updated: 2026-11-05
Code: https://github.com/googledeepmind/mujoco_menagerie
License: http://creativecommons.org/licenses/by/4.0/
The gist: Improvements in multi-modal capabilities have enabled agents to act directly as robot policies, a setting we refer to as agentic robot control (ARC).
Terminology
Abstract
Improvements in multi-modal capabilities have enabled agents to act directly as robot policies, a setting we refer to as agentic robot control (ARC). We introduce RoboARCade, a benchmark of long-horizon manipulation tasks across three embodiments designed to test in-context adaptation, memory, and reasoning for precise control. The tasks build on existing open-source benchmarks, which we make harder by widening the range of initial conditions. RoboARCade is easy to use and extend: all tasks run on a single simulation backend, MuJoCo, each task is a self-contained data pack, and agents connect through interchangeable interfaces, so new tasks, agents, and interfaces can be added independently. The benchmark separates leading agents: GPT-6 Astra achieves 49.8% compared with 43.6% for Claude Fable 5.1, leaving substantial room for improvement. Comparing two interfaces, we find that a richer visual interface adds only 1.5 to 6.8 points at 1.7 to 2.3 times the cost. Failures stem mostly from choosing a poor manipulation strategy or executing a good one imprecisely, rarely from misunderstanding the scene, and agents incorrectly claim success on 24% to 39% of attempted subtasks. Agents can also improve without human input: one successful demonstration in context raises Astra's score from 53.5% to 73.0% on a five-task subset, and a prompt the agent writes and revises itself reaches 72.0%. Code, videos and full traces are available at roboarcade.org.
Sources
- GEPA: Reflective Prompt Evolution Can Outperform Reinforcement Learning
- RT-H: Action Hierarchies Using Language
- RoboTwin 2.0: A Scalable Data Generator and Benchmark with Strong Domain Randomization for Robust Bimanual Robotic Manipulation
- RoboDojo: A Unified Sim-and-Real Benchmark for Comprehensive Evaluation of Generalist Robot Manipulation Policies
- RoboCerebra: A Large-scale Benchmark for Long-horizon Robotic Manipulation Evaluation
- VIA: Visual Interface Agent for Robot Control
- What Matters in Orchestrating Robot Policies: A Systematic Study of Hierarchical VLA Agents
- VisualWebArena: Evaluating Multimodal Agents on Realistic Visual Web Tasks
- Demonstration-Free Robotic Control via LLM Agents
- Agentic Context Engineering: Evolving Contexts for Self-Improving Language Models
- An Unexpected Robot Policy: Early Evaluations of GPT-6 Astra on RoboDojo and Beyond
- robosuite: A Modular Simulation Framework and Benchmark for Robot Learning
Related papers
- Rethinking Actions: Continuity-Aware Priors for World-Model Planning
- MOSAIC: Modular Obstacle and Spatial-Aware Framework for Manipulation In Cluttered Scenes
- SAGAS: Semantic-Aware Graph-Assisted Stitching for Offline Temporal Logic Planning
- DexSeed: Expanding Human Hand-Object Interactions into Robot Demonstrations for Dexterous Policy Learning
- LaST0.5: A Generalist World Action Model with Interaction-Centric Latent Reasoning
- CIALLO: Learning Shared Intent Representations for Cross-Embodiment Manipulation