GuardrailAgent: An Agentic Framework for Jailbreak Defense
cs.CR
Submitted: 2026-11-06
Updated: 2026-11-06
Code: https://github.com/TrustAIRLab/GuardrailAgent
Terminology
Sources
- PUZZLED: Jailbreaking LLMs through Word-Based Puzzles
- Detecting Language Model Attacks with Perplexity
- Jailbreaking Leading Safety-Aligned LLMs with Simple Adaptive Attacks
- A General Language Assistant as a Laboratory for Alignment
- Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback
- A Wolf in Sheep's Clothing: Generalized Nested Jailbreak Prompts can Fool Large Language Models Easily
- Bypassing LLM Guardrails: An Empirical Analysis of Evasion Attacks against Prompt Injection and Jailbreak Detection Systems
- Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversations
- Baseline Defenses for Adversarial Attacks Against Aligned Language Models
- WildTeaming at Scale: From In-the-Wild Jailbreaks to (Adversarially) Safer Language Models
- GUARD: Role-playing to Generate Natural-language Jailbreakings to Test Guideline Adherence of Large Language Models
- Certifying LLM Safety against Adversarial Prompting
- H-CoT: Hijacking the Chain-of-Thought Safety Reasoning Mechanism to Jailbreak Large Reasoning Models, Including OpenAI o1/o3, DeepSeek-R1, and Gemini 2.0 Flash Thinking
- StructuralSleight: Automated Jailbreak Attacks on Large Language Models Utilizing Uncommon Text-Organization Structures
- Faster-GCG: Efficient Discrete Optimization Jailbreak Attacks against Aligned Large Language Models
- DrAttack: Prompt Decomposition and Reconstruction Makes Powerful LLM Jailbreakers
- AutoDAN: Generating Stealthy Jailbreak Prompts on Aligned Large Language Models
- FlipAttack: Jailbreak LLMs via Flipping
- From LLMs to MLLMs to Agents: A Survey of Emerging Paradigms in Jailbreak Attacks and Defenses within LLM Ecosystem
- Tree of Attacks: Jailbreaking Black-Box LLMs Automatically
Related papers
- From Strategy Search to Policy Learning: MCTS-Guided Adversarial Co-Evolution for LLM Safety Alignment
- Privacy Leakage from Recurrent State in LLM Inference
- Incrementally Verifiable Oracle Aided Computation
- Privilege and confidentiality in generative AI workflows
- BadMoE: Backdooring Mixture-of-Experts LLMs via Routing-Aware Triggers and Task-Underutilized Experts
- CoVault: Secure, Scalable Analytics of Personal Data