Strengthening Target-Language Features: SAE-Based Steering for Multilingual Inference

arXiv:2608.04904 · cs.CL · Submitted 2026-08-05 · Read on arXiv

cs.CL

Submitted: 2026-08-05

Updated: 2026-10-11

Comments: Corrected an author name. No changes to the paper content

Code: https://github.com/HungsingWong/sae-languagesteering

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Terminology

• Steering — 68× in this paper · explained in 2 episodes
• Accuracy — 33× in this paper · explained in 1 episode
• Transformer — 13× in this paper · explained in 2 episodes
• Ablation — 7× in this paper · explained in 1 episode
• Papers — 5× in this paper · explained in 3 episodes
• Ablation Study — 3× in this paper · explained in 3 episodes
• Benchmark — 3× in this paper · explained in 2 episodes
• Decoder — 3× in this paper · explained in 2 episodes
• Sparse Autoencoders (SAEs) — 3× in this paper · explained in 4 episodes
• Adam — 2× in this paper · explained in 3 episodes
• Cross-lingual Transfer — 2× in this paper · explained in 2 episodes
• Decode — 2× in this paper · explained in 2 episodes
• Feature Selection — 2× in this paper · explained in 4 episodes
• Inference-Time Steering — 2× in this paper · explained in 4 episodes
• Transformers — 2× in this paper · explained in 2 episodes
• Activation Steering — 1× in this paper · explained in 7 episodes
• Clarity — 1× in this paper · explained in 3 episodes
• Generalization — 1× in this paper · explained in 14 episodes
• Generative AI — 1× in this paper · explained in 2 episodes
• In-Context Learning — 1× in this paper · explained in 7 episodes
• Instruction Tuning — 1× in this paper · explained in 3 episodes
• Language Models (LLMs) — 1× in this paper · explained in 2 episodes
• Large Language Models (LLMs) — 1× in this paper · explained in 9 episodes
• Representation Engineering — 1× in this paper · explained in 2 episodes
• Representation Learning — 1× in this paper · explained in 3 episodes
• Trace — 1× in this paper · explained in 2 episodes

Sources

• Causal Language Control in Multilingual Transformers via Sparse Feature Steering
• No Language Left Behind: Scaling Human-Centered Machine Translation
• Sparse Autoencoders Find Highly Interpretable Features in Language Models
• Language Steering for Multilingual In-Context Learning
• Do Multilingual LLMs Think In English?
• Language Models are Multilingual Chain-of-Thought Reasoners
• Gemma 3 Technical Report
• Steering Language Models With Activation Engineering
• Representation Engineering: A Top-Down Approach to AI Transparency

Related papers

• Exploring Solution Divergence and Its Effect on Large Language Model Problem Solving
• Ishigaki-IDS-Bench: A Benchmark for Generating Information Delivery Specification from BIM Information Requirements
• Subliminal Steering: Stronger Encoding of Hidden Signals
• MedStruct-S: A Benchmark for Key Discovery, Key-Conditioned QA and Semi-Structured Extraction from OCR Clinical Reports
• The End of Transformers? On Challenging Attention and the Rise of Sub-Quadratic Architectures
• Untangling the Mechanisms of Misleading Context in Medical Question Answering