Adversarial Images Hijack Web Agents from Visual Grounding to Browser Execution
cs.CR, cs.AI, cs.CL
Submitted: 2026-10-07
Updated: 2026-10-07
Code: https://github.com/MoonTea0416/WebMirage
Terminology
Sources
- Qwen2 Technical Report
- GPT-4 Technical Report
- WebArena: A Realistic Web Environment for Building Autonomous Agents
- EIA: Environmental Injection Attack on Generalist Web Agents for Privacy Leakage
- WIPI: A New Web Threat for LLM-Driven Web Agents
- Environmental Injection Attacks against GUI Agents in Realistic Dynamic Environments
- WebGPT: Browser-assisted question-answering with human feedback
- WebShop: Towards Scalable Real-World Web Interaction with Grounded Language Agents
- Set-of-Mark Prompting Unleashes Extraordinary Visual Grounding in GPT-4V
- Attacking Vision-Language Computer Agents via Pop-ups
- AdInject: Real-World Black-Box Attacks on Web Agents via Advertising Delivery
- SecureWebArena: A Holistic Security Evaluation Benchmark for LVLM-based Web Agents
- WebInject: Prompt Injection Attack to Web Agents
- An Image Is Worth 1000 Lies: Adversarial Transferability across Prompts on Vision-Language Models
- Jailbreak in pieces: Compositional Adversarial Attacks on Multi-Modal Language Models
- GPT-4o System Card
- MiniCPM-V: A GPT-4V Level MLLM on Your Phone
- CogAgent: A Visual Language Model for GUI Agents
- VILA: On Pre-training for Visual Language Models
- OpenFlamingo: An Open-Source Framework for Training Large Autoregressive Vision-Language Models
Related papers
- SoK: AI-Augmented Binary Reversing
- Relaxed Sender Anonymity for CBDC Interbank Settlement: A Zero-Knowledge Approach on Permissioned EVM
- Calibration-Family Overfit: Why Trusted Sabotage Monitors Don't Transfer Across Lineages
- Efficient Fuzzy PSI under One-Sided Assumptions
- Sealing the Audit-Runtime Gap for LLM Skills
- Token Composition: A Graph Based on EVM Logs