Neither Here Nor There: Cross-Lingual Representation Dynamics of Code-Mixed Text in Multilingual Encoders
The study investigated how multilingual models represent code-mixed text by analyzing their internal representations. They found that pretraining on code-mixed data improves English–code-mixed alignment but hurts English–Hindi alignment. A new trilingual post-training stage, using a specific objective, was introduced to achieve more balanced cross-lingual alignment across English, Hindi, and code-mixed inputs.