Addressing Moral Uncertainty using Large Language Models for Ethical Decision-Making
Rohit K. Dubey, Damian Dailisan, Sachit Mahajan
cs.CY, cs.AI
Submitted: 2025-10-01
Updated: 2026-08-24
Comments: 13 pages, 5 figures. All authors contributed equally to this work
Journal ref: Front. Artif. Intell., 9 (2026)
DOI: 10.3389/frai.2026.1754973
Code: https://github.com/xxxxx/moral
License: http://creativecommons.org/licenses/by-nc-nd/4.0/
The gist: We present an ethical decision-making framework that refines a pre-trained reinforcement learning (RL) model using a task-agnostic ethical layer.
Terminology
Abstract
We present an ethical decision-making framework that refines a pre-trained reinforcement learning (RL) model using a task-agnostic ethical layer. Following initial training, the RL model undergoes ethical fine-tuning, where human feedback is replaced by feedback generated from a large language model (LLM). The LLM embodies consequentialist, deontological, virtue, social justice, and care ethics as moral principles to assign belief values to recommended actions during ethical decision-making. An ethical layer aggregates belief scores from multiple LLM-derived moral perspectives using Belief Jensen-Shannon Divergence and Dempster-Shafer Theory into probability scores that also serve as the shaping reward, steering the agent toward choices that align with a balanced ethical framework. This integrated learning framework helps the RL agent navigate moral uncertainty in complex environments and enables it to make morally sound decisions across diverse tasks. Our approach, tested across different LLM variants and compared with other belief aggregation techniques, demonstrates improved consistency, adaptability, and reduced reliance on handcrafted ethical rewards. This method is especially effective in dynamic scenarios where ethical challenges arise unexpectedly, making it well-suited for real-world applications.
Sources
- Reinforcement Learning Under Moral Uncertainty
- A Formalization of Kant's Second Formulation of the Categorical Imperative
- Generative Agents: Interactive Simulacra of Human Behavior
- The Moral Turing Test: Evaluating Human-LLM Alignment in Moral Decision-Making
- Navigating LLM Ethics: Advancements, Challenges, and Future Directions
- FIRE: Fact-checking with Iterative Retrieval and Verification
- A Review of Safe Reinforcement Learning: Methods, Theory and Applications
- Proximal Policy Optimization Algorithms
Related papers
- GenAIT: Development and Validation of an Objective Generative AI Literacy Test for High School Students
- Reasoning Enhances Robustness to Prompt Injection in LLM-Based Consensus
- Generative AI Purpose-built for Social and Mental Health: A Real-World Pilot
- Comparative analysis of privacy-preserving open-source LLMs regarding extraction of diagnostic information from clinical CMR imaging reports
- PersonaMem-v3: Toward Omni-Platform Personal Intelligence for Holistic User Understanding, Recommendation, and Agentic Tasks
- What is an intelligent system?