A Fair Objective for Human-Empowerment-Preserving AI: Desiderata, Design, and Likely Behavioral Consequences

arXiv:2608.08240 · cs.AI, cs.GT, cs.MA, econ.TH · Submitted 2026-08-08 · Read on arXiv

Jobst Heitzig, Ram Potham

cs.AI, cs.GT, cs.MA, econ.TH

Submitted: 2026-08-08

Updated: 2026-08-11

Comments: Slightly extended version of paper accepted for Algorithmic Decision Theory 2026

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Terminology

• World Model — 17× in this paper · explained in 8 episodes
• Axioms — 12× in this paper · explained in 2 episodes
• AI Agents — 6× in this paper · explained in 2 episodes
• Reinforcement Learning — 5× in this paper · explained in 4 episodes
• Composition — 4× in this paper · explained in 2 episodes
• Objective Function — 4× in this paper · explained in 2 episodes
• Power Distribution — 3× in this paper · explained in 2 episodes
• Adam — 2× in this paper · explained in 3 episodes
• Clustering — 2× in this paper · explained in 2 episodes
• Monotonicity — 2× in this paper · explained in 2 episodes
• Mutual Information — 2× in this paper · explained in 2 episodes
• Papers — 2× in this paper · explained in 3 episodes
• AI Alignment — 1× in this paper · explained in 2 episodes
• Actor — 1× in this paper · explained in 3 episodes
• Foundation Model — 1× in this paper · explained in 3 episodes
• Foundation Models — 1× in this paper · explained in 8 episodes
• Grounding — 1× in this paper · explained in 3 episodes
• K-means Clustering — 1× in this paper · explained in 3 episodes
• Kullback–Leibler Divergence — 1× in this paper · explained in 2 episodes
• Reachability Analysis — 1× in this paper · explained in 2 episodes
• Semantic Understanding — 1× in this paper · explained in 3 episodes

Sources

• Superintelligent Agents Pose Catastrophic Risks: Can Scientist AI Offer a Safer Path?
• Safety from Honesty in a Disinterested AI Predictor
• A Survey of World Models for Autonomous Driving
• World Models
• Unbiased Canonical Set-Valued Oracles Via Lattice Theory
• Penalizing side effects using stepwise relative reachability
• A theory of appropriateness with applications to generative artificial intelligence
• Corrigibility as a Singular Target: A Vision for Inherently Reliable Foundation Models
• Is Sora a World Simulator? A Comprehensive Survey on General World Models and Beyond

Related papers

• MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
• Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
• Agents' Last Exam
• The Clinician's Veto: Navigating Trust, Liability, and Uncertainty in Autonomous AI Prescribing
• MindHelper: Closed-Loop Embodied Mental-State Reasoning for Precision Intervention
• Incumbent Advantage: Brand Bias and Cognitive Manipulation Dynamics in LLM Recommendation Systems