Towards Shutdownable Agents via Stochastic Choice
cs.AI
Submitted: 2024-06-30
Updated: 2026-11-05
Terminology
Sources
- 'Indifference' methods for managing agent rewards
- Managing extreme AI risks amid rapid progress
- Settling the Reward Hypothesis
- Weak-to-Strong Generalization: Eliciting Strong Capabilities With Weak Supervision
- Is Power-Seeking AI an Existential Risk?
- The Off-Switch Game
- Rainbow: Combining Improvements in Deep Reinforcement Learning
- Corrigibility with Utility Preservation
- Risks from Learned Optimization in Advanced Machine Learning Systems
- Incompleteness, Independence, and Negative Dominance
- AI Safety Gridworlds
- The Effects of Reward Misspecification: Mapping and Mitigating Misaligned Models
- Goal Misgeneralization: Why Correct Specifications Aren't Enough For Correct Goals
- Shutdownable Agents through POST-Agency
- A Game-Theoretic Analysis of the Off-Switch Game
Related papers
- Multi-Objective Structured Pruning of LLMs for Latency and Model Size Optimization
- MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
- Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
- Embeddings for Preferences, Not Semantics
- Can We Trust Item Response Theory for AI Evaluation?
- Diagnosing Task Insensitivity in Language Agents