Mitigating loss of control in advanced AI systems through instrumental goal trajectories
cs.AI, cs.CY
Submitted: 2026-02-02
Updated: 2026-02-02
DOI: 10.1007/s44163-026-01856-2
License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
Terminology
Sources
- Constitutional AI: Harmlessness from AI Feedback
- Is Power-Seeking AI an Existential Risk?
- Deceptive Alignment Monitoring
- Contextual Moral Value Alignment Through Context-Based Aggregation
- A Review of the Evidence for Existential Risk from AI via Misaligned Power-Seeking
- Evaluating the Paperclip Maximizer: Are RL-Based Language Models More Likely to Pursue Instrumental Goals?
- Adaptive Preference Scaling for Reinforcement Learning with Human Feedback
- Self-Improvement in Language Models: The Sharpening Mechanism
- AI Alignment: A Comprehensive Survey
- Evaluating Language-Model Agents on Realistic Autonomous Tasks
- Scalable agent alignment via reward modeling: a research direction
- AI Alignment through Reinforcement Learning from Human Feedback? Contradictions and Limitations
- Fully Autonomous AI Agents Should Not be Developed
- The Effects of Reward Misspecification: Mapping and Mitigating Misaligned Models
- Evaluating Frontier Models for Dangerous Capabilities
- AgentBreeder: Mitigating the AI Safety Risks of Multi-Agent Scaffolds via Self-Improvement
- Goal Misgeneralization: Why Correct Specifications Aren't Enough For Correct Goals
- Large Language Model Alignment: A Survey
- Mind the Gap: Examining the Self-Improvement Capabilities of Large Language Models
- Optimal Policies Tend to Seek Power
Related papers
- MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
- Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
- Agents' Last Exam
- The Clinician's Veto: Navigating Trust, Liability, and Uncertainty in Autonomous AI Prescribing
- MindHelper: Closed-Loop Embodied Mental-State Reasoning for Precision Intervention
- Incumbent Advantage: Brand Bias and Cognitive Manipulation Dynamics in LLM Recommendation Systems