Metanormative Theory for RL-Based Moral Agents

arXiv:2608.08220 · cs.AI · Submitted 2026-08-08 · Read on arXiv

Aleks Knoks, Marija Slavkovik

cs.AI

Submitted: 2026-08-08

Updated: 2026-08-11

Comments: The 14th International Workshop on Engineering Multi-Agent Systems (EMAS 2026) held May 25-26, 2026 Co-located with AAMAS 2026 Paphos, Cyprus

License: http://creativecommons.org/licenses/by/4.0/

The gist: The overlapping disciplines of machine ethics and value alignment are concerned with designing artificial agents that are aligned with human values and that act in ethically acceptable ways.

Terminology

Abstract

The overlapping disciplines of machine ethics and value alignment are concerned with designing artificial agents that are aligned with human values and that act in ethically acceptable ways. A recent trend in these disciplines is the use of reinforcement learning (RL) to design such agents, sidelining the philosophical literature that used to play a more central role. Against this backdrop, this paper pursues two goals. The first is to draw out ideas from recent work in metanormative theory that can be useful for designing artificial moral and value-aligned agents. The second is to examine the RL architecture through the lens of these ideas. This will give us clearer criteria for when an RL agent's behavior can be classified as moral, as well as a basis for evaluating and comparing different RL-based approaches to machine ethics and value alignment.

Sources

Related papers