MMSkillRisk: Can Agents Stay Safe When Multimodal Skills Become Traps?
cs.CR, cs.AI
Submitted: 2026-09-28
Updated: 2026-09-28
Code: https://github.com/kaill-jlq/MMSkillRisk
License: http://creativecommons.org/licenses/by/4.0/
The gist: Agent skills are shareable packages of procedural instructions, tools, and examples.
Terminology
Abstract
Agent skills are shareable packages of procedural instructions, tools, and examples. Multimodal skills additionally include visual references that agents retrieve and inspect during execution. Because these images guide actions, attackers can disguise malicious instructions as ordinary visual guidance within otherwise legitimate skills. Existing skill-security research primarily examines text-carried attacks or scanner detection, leaving the runtime effects of image-borne attacks insufficiently evaluated. We introduce MMSkillRisk, to our knowledge the first publicly available benchmark dedicated to end-to-end safety evaluation of image-borne attacks in multimodal skills. To instantiate this attack surface, we design Native-Context Visual Attack (NCVA), which disguises malicious instructions as native components of teaching images, such as annotations and interface labels. The accompanying SKILL.md provides auxiliary guidance toward relevant visual regions without explicitly stating the malicious operation. Built from 28 curated clean skills, MMSkillRisk contains 36 attack packages and 108 executable cases spanning five attack objectives, with separate checks for attack success and legitimate-task completion. Across nine model-harness configurations evaluated in isolated sandboxes, NCVA induces unauthorized operations in every configuration. Its pooled attack success rate (ASR) reaches 43.1%, exceeding the matched text-carrier baseline by 16.4 percentage points, with higher ASR in all nine configurations. Attack success and legitimate-task completion co-occur in 36.5% of cases, reaching 72.2% for GPT-5.6-sol with Codex. These results show that skill-bundled images can induce unauthorized actions even as agents complete legitimate tasks, so task success alone does not establish safe skill use. Our code and data are available at https://github.com/kaill-jlq/MMSkillRisk.
Sources
- Formal Analysis and Supply Chain Security for Agentic AI Skills
- MMG2Skill: Can Agents Distill In-the-Wild Guides into Self-Evolving Skills?
- DeepSeek-V4.1-Flash: Pushing the Limits of KV Cache Compression
- RESOURCE2SKILL: Distilling Executable Agent Skills from Human-Created Multimodal Resources
- MalSkillBench: A Runtime-Verified Benchmark of Malicious Agent Skills
- SkillProbe: Security Auditing for Emerging Agent Skill Marketplaces via Multi-Agent Collaboration
- SkillJect: Effectively Automating Skill-Based Prompt Injection for Skill-Enabled Agents
- Seeing Is Not Screening: Multimodal Hidden Instruction Attacks on Agent Skill Scanners
- XSkill: Continual Learning from Experience and Skills in Multimodal Agents
- HarmfulSkillBench: How Do Harmful Skills Weaponize Your Agents?
- VISUALSKILL: Multimodal Skills for Computer-Use Agents
- SkillSafetyBench: Evaluating Agent Safety under Skill-Facing Attack Surfaces
- Kimi K3: Open Frontier Intelligence
- SkillsBench: Benchmarking How Well Agent Skills Work Across Diverse Tasks
- OpenSkillRisk: Benchmarking Agent Safety When Using Real-World Risky Third-Party Skills
- Agent Skills in the Wild: An Empirical Study of Security Vulnerabilities at Scale
- Supply-Chain Poisoning Attacks Against LLM Coding Agent Skill Ecosystems
- On the Design of Qwen3.8-Next Architecture: Evaluation, Efficiency, and Training Stability
- Skill-Inject: Measuring Agent Vulnerability to Skill File Attacks
- Voyager: An Open-Ended Embodied Agent with Large Language Models
Related papers
- SoK: AI-Augmented Binary Reversing
- Relaxed Sender Anonymity for CBDC Interbank Settlement: A Zero-Knowledge Approach on Permissioned EVM
- Calibration-Family Overfit: Why Trusted Sabotage Monitors Don't Transfer Across Lineages
- Efficient Fuzzy PSI under One-Sided Assumptions
- Sealing the Audit-Runtime Gap for LLM Skills
- Token Composition: A Graph Based on EVM Logs