StepJack: Benchmarking Computer-Use Agent Safety Against Multi-Step Indirect Prompt Injection

arXiv:2608.06477 · cs.CR, cs.AI, cs.CL · Submitted 2026-08-06 · Read on arXiv

Zhuoxin Zhan, Akbar Rafiey, Avery Ma, Leila Pishdad, Layla El Asri

cs.CR, cs.AI, cs.CL

Submitted: 2026-08-06

Updated: 2026-08-10

Code: https://github.com/BorealisAI/StepJack

Project page: https://os-world.github.io/https://github.com/xlang-ai/OSWorld/tree/main/mm_agents

License: http://creativecommons.org/licenses/by-nc-sa/4.0/

The gist: Computer-use agents (CUAs) face a growing threat from indirect prompt injection, where adversarial instructions are planted in the environment such as web pages.

Terminology

Abstract

Computer-use agents (CUAs) face a growing threat from indirect prompt injection, where adversarial instructions are planted in the environment such as web pages. In this paper, we introduce multi-step indirect prompt injection, a new attack class against CUAs in which the adversarial goal is decomposed into multiple innocuous-looking sub-steps and distributed across a chain of pages referenced along the agent's navigation path. We develop a pipeline to automatically decompose an adversarial goal under the constraint that the execution of the decomposed sub-steps must achieve the original goal while optimizing the innocuousness of each decomposed sub-step. With this pipeline, we build StepJack, a CUA safety benchmark with 480 test examples. On this benchmark, we evaluate six state-of-the-art CUAs and find that at a fixed decomposition depth, multi-step attacks raise attack success rate (ASR) on three of six CUAs, by up to 31.2 points (e.g., GPT-5.4-mini: 41.7% at single-step to 72.9% at three-step); averaged over the five CUAs that can reliably follow the reference chain (all but EvoCUA-32B), ASR rises from 31.3% at single-step to 36.9% at three-step. Dataset and code are available at https://github.com/BorealisAI/StepJack.

Sources

Related papers