Safety Cost of Steering Vectors Is Separable and Reducible
Yuxiao Li, Gjergji Kasneci
cs.CL, cs.LG
Submitted: 2026-08-09
Updated: 2026-08-11
Comments: COLM 2026
Code: https://github.com/yetiiil/steering-safety-cost
License: http://creativecommons.org/licenses/by/4.0/
The gist: Steering vectors are a lightweight tool for controlling LLM behavior.
Terminology
Abstract
Steering vectors are a lightweight tool for controlling LLM behavior. However, emerging evidence shows that steering vectors can unintentionally compromise a model's safety mechanisms and increase compliance with harmful requests, while no effective mitigation yet exists. In this work, we show that this safety degradation arises from a separable component in the vector that disrupts the model's safety mechanisms but contributes little to the steering objective. We identify and remove this safety-degrading component, formulating the task as a constrained optimization problem solved through primal-dual updates, subject to preserving the intended steering effect and bounding false refusal. The resulting solution is both interpretable and surgical: the optimization recovers a single direction whose ablation from the steering vector restores model safety with minimal utility cost. Across models, steering behaviors, and attack suites, including unseen attacks types, our method substantially reduces steering-induced safety degradation while preserving the original steering effect with minimal impact on false refusal. Our method offers a post-hoc correction to steering vectors that mitigates their safety cost, and more broadly, it provides a general recipe for applying activation-level model interventions without paying a safety tax.
Sources
- Improving Steering Vectors by Targeting Sparse Autoencoder Features
- Persona Vectors: Monitoring and Controlling Character Traits in Language Models
- Training Verifiers to Solve Math Word Problems
- Toy Models of Superposition
- Red Teaming Language Models to Reduce Harms: Methods, Scaling Behaviors, and Lessons Learned
- The Llama 3 Herd of Models
- Qwen2.5 Technical Report
- Steering Without Side Effects: Improving Post-Deployment Control of Language Models
- Steering Language Models With Activation Engineering
- All Languages Matter: On the Multilingual Safety of Large Language Models
- Finetuned Language Models Are Zero-Shot Learners
- HuggingFace's Transformers: State-of-the-art Natural Language Processing
- Steering Externalities: Benign Activation Steering Unintentionally Increases Jailbreak Risk for Large Language Models
- Instruction-Following Evaluation for Large Language Models
- Representation Engineering: A Top-Down Approach to AI Transparency
- Universal and Transferable Adversarial Attacks on Aligned Language Models
Related papers
- Exploring Solution Divergence and Its Effect on Large Language Model Problem Solving
- Ishigaki-IDS-Bench: A Benchmark for Generating Information Delivery Specification from BIM Information Requirements
- Subliminal Steering: Stronger Encoding of Hidden Signals
- MedStruct-S: A Benchmark for Key Discovery, Key-Conditioned QA and Semi-Structured Extraction from OCR Clinical Reports
- The End of Transformers? On Challenging Attention and the Rise of Sub-Quadratic Architectures
- Untangling the Mechanisms of Misleading Context in Medical Question Answering