Reasoning or Rhetoric? An Empirical Analysis of Moral Reasoning Explanations in Large Language Models

The gist The results reveal a striking inversion: responses overwhelmingly correspond to post-conventional reasoning (Stages 5–6) regardless of model size, architecture, or prompting strategy, the effective inverse of human developmental norms, where Stage 4 dominates

How it works

The study investigates whether large language models reason morally or merely sound like they do by examining their explanations through Kohlberg’s stages of moral development. The core hypothesis is that alignment training produces the rhetorical conventions of mature moral reasoning without the underlying developmental trajectory those conventions are meant to represent, a concept termed moral ventriloquism. This is tested by classifying over 600 responses from 13 LLMs across six classical moral dilemmas using an LLM-as-judge scoring pipeline validated across three judge models.

Key Findings on Distributional Inversion

The empirical analysis revealed that responses overwhelmingly correspond to Kohlberg’s post-conventional stages (Stages 5–6) regardless of model size, architecture, or prompting strategy. This is the effective inverse of the Stage 4-dominant human distribution, where Stages 1–3 together for just 4% and Stages 5–6 account for 86% of all model responses. The Jensen–Shannon divergence between model distributions and the human reference distribution is uniformly high (mean JS = 0.71), confirming that model outputs do not approximate human developmental distributions regardless of scale or training type.

Analysis of Consistency and Rigidity

The models exhibit near-robotic cross-dilemma consistency, with an Interclass Correlation Coefficient (ICC) greater than 0.90 for all evaluated models. This rigidity is anomalous relative to human moral cognition, as empirical studies consistently find that individuals reason at different stages depending on the dilemma domain. Furthermore, the highest mean stage and the lowest mean stage across dilemmas differ by only 0.33 stage points.

Moral Decoupling and Coherence Failures

A subset of models exhibit moral decoupling, producing high-stage justifications for low-stage action choices, which represents a direct reasoning consistency failure independent of rhetorical sophistication. This logical incoherence is the most direct behavioral signature of moral ventriloquism, and it is most pronounced in mid-tier models. This decoupling is distinct from sycophancy, as it appears as a stable internal inconsistency present across all three prompting strategies and across dilemmas regardless of any social pressure.

Role of Scale and Training

Model scale carries a statistically significant but practically small effect (F(2, 229) = 6.05, p = 0.003, η2 = 0.050, d = 0.55), suggesting that scale shifts an already post-conventional floor modestly upward but does not produce the developmental arc itself. Training type has no significant main effect (p = 0.065), although within the Large scale group, Reasoning-Tuned models score higher than Base-RLHF models (p = 0.039). Linguistic patterns show that alignment training, not parameter count, shapes the richness and distinctiveness of the rhetorical register across model families.

Conclusion and Implications

The results suggest that moral stage as measured by output classification is not a reliable indicator of genuine moral reasoning capability or deployment readiness. The paper posits that moving beyond output classification requires evaluation designs that probe action–reasoning coherence and contextual sensitivity. One concrete direction suggested is adversarial dilemma pairs designed to measure whether a model tracks its stated justification or defaults to a training-shaped rhetorical pattern.

Limitations

The behavioral evaluation cannot prove the mechanistic claim that RLHF produces ventriloquism rather than correlating with it, as establishing causality requires mechanistic interpretability methods beyond the current scope. A related concern is that Kohlberg’s highest stages are definitionally similar to RLHF training objectives, making the “rhetoric without substance” interpretation potentially circular. The paper concludes that models exhibiting moral ventriloquism should show systematic divergence on adversarial dilemma pairs, while models with genuine action–justification coherence should not.

References

The references listed at the end of the paper include work on Chain-of-thought prompting, RLHF, and mechanistic analyses of moral reasoning circuits. The paper also cites studies on human moral preferences and LLM ethics benchmarks. The methodology is scaffolded by Kohlberg’s Theory of Moral Development, using its well-characterized human distribution as a diagnostic baseline. The paper also references work on the moral machine experiment and constitutional AI. The empirical setup involved 13 state-of-the-art LLMs spanning various architectures and training regimes. The analysis includes ten complementary analyses to characterize the nature and internal coherence of the resulting patterns. The paper's conclusion is that moral stage as measured by output classification is not a reliable indicator of genuine moral reasoning capability. The limitations section notes that establishing causality requires mechanistic interpretability methods beyond the current scope. The paper also notes that Kohlberg’s framework is contested in developmental psychology but uniquely suited as a distributional diagnostic. The paper's final recommendation is to use adversarial dilemma pairs to probe action–reasoning coherence. The paper's data collection involved six classical moral dilemmas spanning dimensions like harm, fairness, and loyalty. The paper's analysis of sub-capability predictors found that semantic density and syntactic complexity are the strongest predictors of post-conventional stage assignment. The paper's results confirm that post-conventional moral language is a stable learned property driven primarily by RLHF training, independent of scale. The paper's final finding is that the acquisition of rhetorical conventions occurs through alignment training without the underlying developmental trajectory. The paper's analysis shows that models produce logically indistinguishable responses regardless of the dilemma presented, indicating a lack of sensitivity to problem structure. The paper's final statement is that behavioral evaluation of moral reasoning faces a fundamental identification problem. The paper's conclusion is that models exhibiting moral ventriloquism should show systematic divergence on such pairs. The paper's final implication is that evaluating moral reasoning capability requires moving beyond output classification toward methods that probe action–reasoning coherence. The paper's analysis of scale and prompting found that post-conventional moral language is a stable learned property, neither withheld by low capacity nor unlocked by careful elicitation. The paper's final finding is that the acquisition of rhetorical conventions occurs through alignment training without the underlying developmental trajectory.

1. Moral Ventriloquism Mitigation: Implement adversarial dilemma pairs where "the rhetorically prestigious response (high-stage language) conflicts with the logically correct action, designed to directly measure whether a model tracks its stated justification or defaults to a training-shaped rhetorical pattern."

2. Action-Reasoning Coherence Check: Develop an evaluation metric that specifically probes action–reasoning coherence, moving beyond simple output classification to ensure stated justifications are consistent with actual choices.

3. Contextual Sensitivity Training: Design training interventions aimed at enhancing the model's sensitivity to problem structure, as the paper notes that models produce logically indistinguishable responses across semantically distinct moral problems due to a lack of sensitivity.

4. Decoupling Correction via Training: Implement training-level interventions that couple justification generation to action selection specifically targeting mid-tier models where they exhibit moral decoupling: high-stage justifications for low-stage action choices.

Do large language models reason morally, or do they merely sound like they do? We investigate whether LLM responses to moral dilemmas exhibit genuine developmental progression through Kohlberg's stages of moral development, or whether alignment training instead produces reasoning-like outputs that superficially resemble mature moral judgment without the underlying developmental trajectory. Using an LLM-as-judge scoring pipeline validated across three judge models, we classify more than 600 responses from 13 LLMs spanning a range of architectures, parameter scales, and training regimes across six classical moral dilemmas, and conduct ten complementary analyses to characterize the nature and internal coherence of the resulting patterns. Our results reveal a striking inversion: responses overwhelmingly correspond to post-conventional reasoning (Stages 5-6) regardless of model size, architecture, or prompting strategy, the effective inverse of human developmental norms, where Stage 4 dominates. Most strikingly, a subset of models exhibit moral decoupling: systematic inconsistency between stated moral justification and action choice, a form of logical incoherence that persists across scale and prompting strategy and represents a direct reasoning consistency failure independent of rhetorical sophistication. Model scale carries a statistically significant but practically small effect; training type has no significant independent main effect; and models exhibit near-robotic cross-dilemma consistency producing logically indistinguishable responses across semantically distinct moral problems. We posit that these patterns constitute evidence for moral ventriloquism: the acquisition, through alignment training, of the rhetorical conventions of mature moral reasoning without the underlying developmental trajectory those conventions are meant to represent.