Tail-risk-aware Agent Safety: Constraining Tail Risks Across Multi-step LLM Agent Workflows
cs.AI
Submitted: 2026-11-07
Updated: 2026-11-07
Terminology
Sources
- Concrete Problems in AI Safety
- Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback
- Regret Bounds for Risk-Sensitive Reinforcement Learning
- AgentPoison: Red-teaming LLM Agents via Poisoning Memory or Knowledge Bases
- AgentDojo: A Dynamic Environment to Evaluate Prompt Injection Attacks and Defenses for LLM Agents
- Deliberative Alignment: Reasoning Enables Safer Language Models
- AgentSentinel: An End-to-End and Real-Time Security Defense Framework for Computer-Use Agents
- Large Language Models Cannot Self-Correct Reasoning Yet
- DeepInception: Hypnotize Large Language Model to Be Jailbreaker
- SecurityLingua: Efficient Defense of LLM Jailbreak Attacks via Security-Aware Prompt Compression
- Membership Inference Attacks against Language Models via Neighbourhood Comparison
- Beyond CVaR: Leveraging Static Spectral Risk Measures for Enhanced Decision-Making in Distributional Reinforcement Learning
- Progent: Securing AI Agents with Privilege Control
- The Rise and Potential of Large Language Model Based Agents: A Survey
- GuardAgent: Safeguard LLM Agents by a Guard Agent via Knowledge-Enabled Reasoning
- A-MEM: Agentic Memory for LLM Agents
- Robust LLM safeguarding via refusal feature adversarial training
- FinMem: A Performance-Enhanced LLM Trading Agent with Layered Memory and Character Design
- R-Judge: Benchmarking Safety Risk Awareness for LLM Agents
- Refuse Whenever You Feel Unsafe: Improving Safety in LLMs via Decoupled Refusal Training
Related papers
- Multi-Objective Structured Pruning of LLMs for Latency and Model Size Optimization
- MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
- Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
- Embeddings for Preferences, Not Semantics
- Can We Trust Item Response Theory for AI Evaluation?
- Diagnosing Task Insensitivity in Language Agents