Language Steering for Multilingual In-Context Learning
The method introduces 'language vectors' to improve multilingual in-context learning without retraining models. These vectors are calculated offline by finding the mean activation difference between source and target language examples at specific layers of a large language model. Adding these vectors as an offset during inference steers the model toward target language reasoning patterns, showing that language identity occupies structured directions in the model's activation space.