Multimodal-SEA: Benchmarking AI Agent Security under Multimodal Social Engineering Attacks
cs.CL
Submitted: 2026-11-05
Updated: 2026-11-05
Code: https://github.com/Zhitao-He/M-SEABench
License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
Terminology
Sources
- Defending Against Social Engineering Attacks in the Age of LLMs
- AgentHarm: A Benchmark for Measuring Harmfulness of LLM Agents
- On the Feasibility of Using MultiModal LLMs to Execute AR Social Engineering Attacks
- DoomArena: A framework for Testing AI Agents Against Evolving Security Threats
- JailbreakBench: An Open Robustness Benchmark for Jailbreaking Large Language Models
- DECEPTICON: How Dark Patterns Manipulate Web Agents
- AgentDojo: A Dynamic Environment to Evaluate Prompt Injection Attacks and Defenses for LLM Agents
- Personalized Attacks of Social Engineering in Multi-turn Conversations: LLM Agents for Simulation and Detection
- AgentDyn: Are Your Agent Security Defenses Deployable in Real-World Dynamic Environments?
- FakeBench: Probing Explainable Fake Image Detection via Large Multimodal Models
- MM-SafetyBench: A Benchmark for Safety Evaluation of Multimodal Large Language Models
- HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusal
- Identifying the Risks of LM Agents with an LM-Emulated Sandbox
- When Bots Take the Bait: Exposing and Mitigating the Emerging Social Engineering Attack in Web Automation Agent
Related papers
- ALARM: Audio-Language Alignment for Reasoning Models
- ThinkSum: Probabilistic reasoning over sets using large language models
- Coherence boosting: When your pretrained language model is not paying enough attention
- From Facts to Insights: A Persona-Driven Dual Memory Framework and Dataset for Role-Playing Agents
- Improving General Role-Playing Agents via Psychology-Grounded Reasoning and Role-Aware Policy Optimization
- The Art of Asking: Multilingual Prompt Optimization for Synthetic Data