Online Safety Monitoring for LLMs
Mona Schirmer, Metod Jazbec, Alexander Timans, Christian Naesseth, Maja Waldron, Eric Nalisnick
cs.AI, cs.CL, cs.LG, stat.AP, stat.ML
Submitted: 2026-07-02
Comments: ICML 2026 Hypothesis Testing Workshop
Code: https://github.com/monasch/llm-monitor
License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
Terminology
Sources
- On the Opportunities and Risks of Foundation Models
- Calibrated Predictive Lower Bounds on Time-to-Unsafe-Sampling in LLMs
- How Many Iterations to Jailbreak? Dynamic Budget Allocation for Multi-Turn LLM Evaluation
- Red Teaming Language Models to Reduce Harms: Methods, Scaling Behaviors, and Lessons Learned
- Conformal Risk Control
- AI Control: Improving Safety Despite Intentional Subversion
- Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback
- Monitoring Reasoning Models for Misbehavior and the Risks of Promoting Obfuscation
- Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversations
- Efficiently Controlling Multiple Risks with Pareto Testing
- From Judgment to Interference: Early Stopping LLM Harmful Outputs via Streaming Content Monitoring
- Mistral 7B
- WildTeaming at Scale: From In-the-Wild Jailbreaks to (Adversarially) Safer Language Models
- HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusal
- Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learning
- Language Models with Conformal Factuality Guarantees
- Agentic Uncertainty Reveals Agentic Overconfidence
- Tracking the risk of a deployed model and detecting harmful distribution shifts
- Chain of Thought Monitorability: A New and Fragile Opportunity for AI Safety
- Conformal Language Modeling
Related papers
- MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
- Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
- Agents' Last Exam
- The Clinician's Veto: Navigating Trust, Liability, and Uncertainty in Autonomous AI Prescribing
- MindHelper: Closed-Loop Embodied Mental-State Reasoning for Precision Intervention
- Incumbent Advantage: Brand Bias and Cognitive Manipulation Dynamics in LLM Recommendation Systems