Who Bridges Safety? Identifying and Targeting Cross-Lingual Shared Safety Pathways

The paper Who Bridges Safety? Identifying and Targeting Cross-Lingual Shared Safety Pathways investigates the internal mechanisms behind the cross-lingual safety gap in large language models (LLMs), where safety behaviors are more reliable in high-resource (HR) languages than in non-high-resource (NHR) languages. The authors move beyond isolated neuron analysis to identify cross-layer functional pathways formed during safety signal propagation.

The paper's abstract states: "We move beyond isolated neurons to identify and target the cross-layer functional pathways formed during safety signal propagation, thereby uncovering the mechanisms driving the cross-lingual safety gap. Specifically, we first identify monolingual safety pathways and validate their impact on refusing harmful requests. Subsequent cross-lingual analyses reveal a sparse subset of cross-lingual shared safety pathways, confirming that this intersection acts as the internal bridge transferring safety capabilities from high-resource (HR) languages to non-high-resource (NHR) languages."

The methodology involves three main steps. First, the authors identify monolingual safety pathways by localizing safety neurons using a gradient-activation attribution score and then modeling cross-layer connections through two complementary views: co-activation pathways and activation-propagation pathways. The co-activation pathways capture coordinated activation of safety neurons across adjacent layers, while activation-propagation pathways capture directed cross-layer dependencies through causal interventions. The authors validate these pathways by showing that masking them causes significantly higher attack success rates (ASR) than masking random pathways.

Second, the authors identify cross-lingual shared safety pathways by examining the overlap between HR and NHR language safety pathways, defining them as Pλ⋆ = Pλ ∩ PHR. They verify that masking these shared pathways causes substantial ASR increases across NHR languages, confirming they serve as a critical bridge through which HR-aligned refusal behaviour may be transferred to NHR languages.

Third, based on these mechanistic findings, the authors propose a pathways-targeted cross-lingual alignment method. This method constructs a cross-lingual safety training set by translating harmful queries and correct answers from HR to NHR languages, then exclusively updates parameters associated with safety pathways while freezing all other parameters. The update rule is: θt+1 = θt − η m ⊙ ∇θ L(Dtrain), where m is a binary parameter mask retaining gradients only for safety pathway parameters.

The experimental results demonstrate the effectiveness of this approach. On Gemma-2-9B-it, the method restricts average ASR to 0.19% on AdvBench-x and 2.29% on MultiJail, outperforming the strongest baseline by 0.81% and 1.01% respectively. On Llama-3.1-8B-it, it yields average ASRs of 1.00% and 4.58%, reducing ASR by 0.81% and 0.69% relative to the best baseline. The method achieves these results while updating only 0.46%–0.71% of parameters, and largely preserves general capabilities on MGSM and MMMLU benchmarks.

Ablation studies confirm that combining both co-activation and activation-propagation pathways is necessary, and that pathway-level targeting outperforms neuron-level targeting. The paper also demonstrates that the method recruits additional safety pathways across languages, particularly in NHR languages, and strengthens activation of shared safety pathways. Leave-one-language-out experiments show zero-shot transfer to unseen NHR languages, with average ASR dropping from 3.66% to 0.66% on AdvBench-x and from 10.10% to 3.87% on MultiJail.

The paper concludes: "We find that safety capabilities are supported by safety pathways formed as safety information propagates across layers. Further cross-lingual analysis reveals safety pathways shared between HR and NHR languages, which serve as internal bridges for transferring safety capabilities. Based on this finding, we propose a pathways-targeted cross-lingual alignment that facilitates the transfer of safety from HR to NHR languages by fine-tuning only a very small fraction of the parameters associated with safety pathways."

Improvements to AI Systems:

1. Cross-Lingual Safety Transfer via Pathway-Targeted Fine-Tuning

• Implement a parameter-efficient alignment method that identifies and updates only the cross-lingual shared safety pathways (0.46%–0.71% of parameters) instead of full fine-tuning. This enables rapid, low-cost safety hardening for new NHR languages without degrading general capabilities.

2. Mechanistic Safety Auditing

• Add a diagnostic tool that maps safety pathways (co-activation and activation-propagation) for any given language. This allows developers to visualize where safety signals break down in NHR languages, enabling targeted interventions rather than trial-and-error prompt engineering.

3. Zero-Shot Safety Generalization

• Use the identified shared pathways to pre-activate safety mechanisms in unseen languages. The leave-one-language-out results (ASR drop from 3.66% to 0.66% on AdvBench-x) show that models can be pre-configured to refuse harmful requests in new languages without any fine-tuning, just by aligning pathway activations.

4. Dynamic Safety Pathway Reinforcement

• During inference, monitor activation strength of shared safety pathways. If activation weakens (e.g., due to adversarial input or domain shift), trigger a corrective intervention—such as re-routing through HR-language safety pathways—to restore refusal behavior in real time.

5. Cross-Lingual Safety Training Data Generation

• Automatically generate high-quality safety training sets by translating harmful queries and correct refusals from HR to NHR languages, then apply pathway-targeted updates. This reduces the need for manual NHR safety data curation, which is often scarce and biased.

What the Improved AI System Can Do:

• Maintain near-zero attack success rates (e.g., 0.19% on AdvBench-x) across multiple NHR languages while updating less than 1% of parameters.

• Instantly extend safety to new languages with zero-shot transfer, requiring no additional training data or compute.

• Provide interpretable safety diagnostics—developers can see exactly which cross-layer pathways are active or missing, and why a model fails to refuse in a specific language.

• Adaptively defend against novel attacks by reinforcing shared safety pathways on-the-fly, without full retraining.

• Preserve general capabilities (MGSM, MMMLU) while improving safety, avoiding the typical trade-off seen in full fine-tuning approaches.