SIGMA: Self-Improving Alignment Generalization from a Model Spec
cs.AI
Submitted: 2026-10-06
Updated: 2026-10-06
Code: https://github.com/UKGovernmentBEIS/inspect_ai
License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
Terminology
Sources
- AgentHarm: A Benchmark for Measuring Harmfulness of LLM Agents
- Constitutional AI: Harmlessness from AI Feedback
- Monitoring Reasoning Models for Misbehavior and the Risks of Promoting Obfuscation
- Measuring Progress on Scalable Oversight for Large Language Models
- Reasoning Models Don't Always Say What They Think
- Automated Researchers Can Mitigate Well-characterized Alignment Failures
- Supervising strong learners by amplifying weak experts
- DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning
- Alignment faking in large language models
- Deliberative Alignment: Reasoning Enables Safer Language Models
- Evaluating the Paperclip Maximizer: Are RL-Based Language Models More Likely to Pursue Instrumental Goals?
- AI safety via debate
- Reinforcement Learning Towards Broadly and Persistently Beneficial Models
- Debating with More Persuasive LLMs Leads to More Truthful Answers
- Autodata: An agentic data scientist to create high quality synthetic data
- Specific versus General Principles for Constitutional AI
- Scalable agent alignment via reward modeling: a research direction
- Decomposing and Measuring Evaluation Awareness
- Model Spec Midtraining: Improving How Alignment Training Generalizes
- DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models
Related papers
- MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
- Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
- Agents' Last Exam
- The Clinician's Veto: Navigating Trust, Liability, and Uncertainty in Autonomous AI Prescribing
- MindHelper: Closed-Loop Embodied Mental-State Reasoning for Precision Intervention
- Incumbent Advantage: Brand Bias and Cognitive Manipulation Dynamics in LLM Recommendation Systems