A Virtuous AI is an Existential Risk
cs.CY, cs.AI, cs.LG
Submitted: 2026-06-11
Updated: 2026-09-30
Code: https://github.com/safety-research/petri
Sources
- Public Constitutional AI
- Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback
- Constitutional AI: Harmlessness from AI Feedback
- Direction-Flipped Influence Audits Reveal Hidden Structure in Moral Choices of LLMs
- Is Power-Seeking AI an Existential Risk?
- DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligence
- Deliberative Alignment: Reasoning Enables Safer Language Models
- An Overview of Catastrophic AI Risks
- Specific versus General Principles for Constitutional AI
- Model Spec Midtraining: Improving How Alignment Training Generalizes
- Taking AI Welfare Seriously
- Agentic Misalignment: How LLMs Could Be Insider Threats
- The Alignment Problem from a Deep Learning Perspective
- Discovering Language Model Behaviors with Model-Written Evaluations
Related papers
- Reasoning Enhances Robustness to Prompt Injection in LLM-Based Consensus
- Generative AI Purpose-built for Social and Mental Health: A Real-World Pilot
- PersonaMem-v3: Toward Omni-Platform Personal Intelligence for Holistic User Understanding, Recommendation, and Agentic Tasks
- What is an intelligent system?
- AI University: An LLM-Powered Learning Assistant for Engineering---A Finite Element Method Case Study
- Generative AI Use in Entrepreneurship: An Integrative Review and an Empowerment-Entrapment Framework