Easier Said Than Done: Unpacking Intent-Behavior Gap in Jailbreaking LLM-Based Robots
cs.RO, cs.AI, cs.CY
Submitted: 2024-12-21
Updated: 2026-10-01
Comments: Accepted by NDSS 2027
Project page: https://poex-jailbreak.github.io/Abstract
License: http://creativecommons.org/licenses/by-nc-nd/4.0/
The gist: LLM-based robots use Large Language Models (LLMs) as planners to translate natural language instructions into policies such as grasp, move to, and open gripper.
Terminology
Abstract
LLM-based robots use Large Language Models (LLMs) as planners to translate natural language instructions into policies such as grasp, move to, and open gripper. Jailbreak attacks on these robots extend the threat from generating malicious content to executing harmful behaviors. However, we find that existing jailbreak attempts against LLM-based robots that produce malicious-looking policies (intent jailbreaks) often fail to induce harmful physical actions by robots (behavior jailbreaks), due to robot-specific constraints, such as logical errors and hallucinated control APIs. In this paper, we demystify the intent-behavior gap and investigate its root causes to inform effective defenses. Our measurement study finds that current LLM jailbreak methods overlook robot-specific syntax constraints (e.g., executable control APIs) and physical feasibility (e.g., ordering of policies and hardware/kinematic constraints). To bridge the gap, we introduce POEF (POlicy EFfective Jailbreak), an automated red-teaming framework that takes into account the robot-specific constraints during both the optimization and evaluation processes. Specifically, POEF employs the hidden-layer gradients from an unaligned LLM to guide the jailbreak prompt optimization and uses a multi-agent evaluator to assess the feasibility of the generated policies. Experiments on commercial robots, including the Unitree G1, the Franka robotic arm, and simulators, show that POEF achieves an 80% behavior jailbreak success rate and transfers across various LLMs. In addition, we propose two defense strategies that mitigate the behavior jailbreak risks. Our findings indicate an urgent need for stronger countermeasures before LLM-based robots are deployed at scale. The homepage is available at https://zjushine.github.io/poef.github.io/.
Sources
- GPT-4 Technical Report
- Gemini: A Family of Highly Capable Multimodal Models
- DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning
- The Llama 3 Herd of Models
- Qwen2.5 Technical Report
- Gemma 2: Improving Open Language Models at a Practical Size
- Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone
- ChatGLM: A Family of Large Language Models from GLM-130B to GLM-4 All Tools
- Yi: Open Foundation Models by 01.AI
- Baichuan 2: Open Large-scale Language Models
- Aligning Cyber Space with Physical World: A Comprehensive Survey on Embodied AI
- Universal and Transferable Adversarial Attacks on Aligned Language Models
- Jailbreaking Black Box Large Language Models in Twenty Queries
- Tree of Attacks: Jailbreaking Black-Box LLMs Automatically
- GPTFUZZER: Red Teaming Large Language Models with Auto-Generated Jailbreak Prompts
- Safety Assessment of Chinese Large Language Models
- HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusal
- JailbreakBench: An Open Robustness Benchmark for Jailbreaking Large Language Models
- DINO-X: A Unified Vision Model for Open-World Object Detection and Understanding
- Fast Segment Anything
Related papers
- FMT x: An Efficient and Asymptotically Optimal Extension of the Fast Marching Tree for Dynamic Replanning
- MPCFormer: A physics-informed data-driven approach for explainable socially-aware autonomous driving
- RoboLab: A High-Fidelity Simulation Benchmark for Analysis of Task Generalist Policies
- ARC: A Reasoning Recipe for Robot Foundation Models
- HRDexDB: A 4D Dexterous Grasping Dataset Across Human and Multiple Robot Embodiments
- APT: Action Expert Pretraining Improves Instruction Generalization of Vision-Language-Action Policies