Values as Style: Disentangling Values from Semantics with One-Way Mixing for Low-Damage LLM Steering
cs.AI, cs.CY, cs.LG
Submitted: 2026-09-30
Updated: 2026-09-30
License: http://creativecommons.org/licenses/by-nc-nd/4.0/
The gist: Value steering should change an LLM's normative priorities while preserving the scenario, facts, and task constraints underlying its answer.
Terminology
Abstract
Value steering should change an LLM's normative priorities while preserving the scenario, facts, and task constraints underlying its answer. Conventional activation edits often change both. We introduce an editable semantic-value interface on frozen residual states, with a one-way semantic-to-value pathway that grounds value recognition in context. Stop-gradient blocks feedback through this pathway; swap consistency, topic de-confounding, and decorrelation encourage selective codes. At inference, editing the value code produces a residual delta while holding the semantic code fixed. On two instruction-tuned backbones, this interface improves semantic preservation and reduces benign refusals at comparable value alignment. A matched mixing-by-gating ablation separates representation learning from selective edit activation, and dimension-matched probes establish improved code selectivity. Against validation-selected prompting on LLaMA-3.1-8B, the method achieves comparable alignment (0.750 vs. 0.748), higher BERTScore (0.938 vs. 0.923), and fewer contradictions (5.1% vs. 7.6%). Human ratings and cross-taxonomy controls provide complementary evidence for low-damage value steering.
Sources
- Steering Language Models Before They Speak: Logit-Level Interventions
- Constitutional AI: Harmlessness from AI Feedback
- YaPO: Learnable Sparse Activation Steering Vectors for Domain Adaptation
- Sparse Autoencoders Find Highly Interpretable Features in Language Models
- The Anatomy of Alignment: Decomposing Preference Optimization by Steering Sparse Features
- Steering Language Model Refusal with Sparse Autoencoders
- ValueBench: Towards Comprehensively Evaluating Value Orientations and Understanding of Large Language Models
- Value Kaleidoscope: Engaging AI with Pluralistic Human Values, Rights, and Duties
- Steering Language Models With Activation Engineering
- Measuring Human and AI Values Based on Generative Psychometrics with Large Language Models
- BERTScore: Evaluating Text Generation with BERT
- Representation Engineering: A Top-Down Approach to AI Transparency
- Improving Alignment and Robustness with Circuit Breakers
Related papers
- MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
- Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
- Agents' Last Exam
- The Clinician's Veto: Navigating Trust, Liability, and Uncertainty in Autonomous AI Prescribing
- MindHelper: Closed-Loop Embodied Mental-State Reasoning for Precision Intervention
- Incumbent Advantage: Brand Bias and Cognitive Manipulation Dynamics in LLM Recommendation Systems