BadMoE: Backdooring Mixture-of-Experts LLMs via Routing-Aware Triggers and Task-Underutilized Experts
summary
The gist
Mixture-of-Experts (MoE) models present a new security vulnerability where dormant experts can be poisoned and activated by optimized routing triggers to gain control over model outputs The Core…
In short
BadMoE introduces a backdoor attack targeting Mixture-of-Experts (MoE) models by poisoning dormant experts and using optimized routing triggers to activate them. The attack works in three stages: identifying underutilized experts, optimizing triggers to force their activation, and training the model so these poisoned experts dominate outputs. This demonstrates a new vulnerability in MoE architectures.
Key concepts
- Dormant Expert Probing
- This initial step involves measuring how often each expert is used during standard tasks. Experts with low usage scores are identified as 'dormant' and selected to form the set targeted by the attack. This process helps pinpoint components that are currently inactive in the model's operation.
- Routing-Aware Trigger Optimization
- This stage focuses on finding a specific input trigger that forces the dormant experts to activate. The method uses a loss function designed to align the router's output with a target vector representing these dormant experts, ensuring the trigger is optimized for stealth and effectiveness.
- Dormant Expert Infecting
- The final stage involves training a poisoned dataset where inputs contain the optimized trigger. Crucially, only the parameters of the selected dormant experts are updated during this fine-tuning. This forces these specific experts to become dominant in model responses when triggered.
- Hidden State Separability Defense
- This defense attempts to detect backdoors by clustering hidden states generated by the model. If backdoor implants exist, they often create distinct clusters in the hidden state space, allowing detection even when only a small percentage of training data is poisoned.
Terminology used across episodes
This episode discusses
- BadMoE: Backdooring Mixture-of-Experts LLMs via Routing-Aware Triggers and Task-Underutilized Experts · Paper Radio
- Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone
- GPT-4 Technical Report
- A Survey on Mixture of Experts in Large Language Models
- UOR: Universal Backdoor Attacks on Pre-trained Language Models
- The Llama 3 Herd of Models · Paper Radio
- BadNets: Identifying Vulnerabilities in the Machine Learning Model Supply Chain
- DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning
- Buffer Overflow in Mixture of Experts
- Efficiently Editing Mixture-of-Experts Models with Compressed Experts
- GPT-4o System Card
- Mistral 7B
- Mixtral of Experts
- BackdoorLLM: A Comprehensive Benchmark for Backdoor Attacks and Defenses on Large Language Models
- DeepSeek-V3 Technical Report
- OLMoE: Open Mixture-of-Experts Language Models
- Towards Making the Most of ChatGPT for Machine Translation
- Know What You Don't Know: Unanswerable Questions for SQuAD
- Universal Jailbreak Backdoors from Poisoned Human Feedback
- Rerouting LLM Routers
- Generalizing Motion Planners with Mixture of Experts for Autonomous Driving
The paper
BadMoE: Backdooring Mixture-of-Experts LLMs via Routing-Aware Triggers and Task-Underutilized Experts · Read on arXiv
Hong Kong University of Science and Technology · Carnegie Mellon University · Institute of Information Engineering, Chinese Academy of Sciences
Transcript
Introduction to the show: ident: Security Radio. Generated commentary on the latest security and cryptography papers.
Nadia: We're looking at the paper: "BadMoE: Backdooring Mixture-of-Experts LLMs via Routing-Aware Triggers and Task-Underutilized Experts".
Elias: Here's the gist of the paper. Mixture-of-Experts (MoE) models present a new security vulnerability where dormant experts can be poisoned and activated by optimized routing triggers to gain control over model outputs The Core Attack Mechanism The paper introduces BadMoE, the first backdoor attack specifically designed for MoE-based LLMs, which strategically injects dormant experts that remain inactive during standard inference and activates them by optimizing routing triggers to control the model’s output The core mechanism involves a three-stage pipeline: ❶ Dormant Expert Probing, ❷ Routing-Aware Trigger Optimizing, and ❸ Dormant Expert Infecting The attack exploits the unique architecture of MoE models by identifying dormant experts unrelated to the target task, constructing a routing-aware loss to optimize activation triggers, and promoting dormant experts to dominating roles via poisoned training data Identification of Dormant Experts The initial stage involves measuring expert utility using routing scores from clean data samples to quantify how frequently each expert is selected The authors define a usage score, denoted as r i, which indicates the frequency of activation for the i-th expert across a subset of clean samples Based on these scores, they rank all experts and select the N a experts with the lowest scores to construct the dormant expert set S a This selection process is guided by constructing a binary routing target vector v that sets indices of selected dormant experts to 0 Trigger Optimization Strategy The second stage focuses on finding appropriate triggers that enable the activation of the dormant experts S a through optimization This is achieved by designing a routing-aware loss function, L a, which aligns the router’s output with the target vector v A perplexity-based constraint is incorporated into this optimization to ensure that the trigger remains relatively natural and stealthy The trigger generation is formulated as a minimum optimization problem aimed at minimizing L a while satisfying a perplexity constraint Infection and Dominance The final stage, Dormant Expert Infecting, involves constructing a backdoored training set D star by inserting the optimized trigger z star into clean inputs to form poisoned samples To ensure the dormant experts S a become dominant, all other experts in the targeted MoE layer are frozen, and only the parameters associated with S a are updated This targeted training forces the dormant experts to dominate the model’s output when triggered Experimental Validation and Robustness Extensive experiments on three public MoE-based LLMs, including Mixtral-8x7B, OLMoE-1B-7B, and Deepseek-moe-16B, demonstrate that BadMoE consistently achieves attack success rates (ASR) above 95% across most evaluated tasks The method maintains competitive or superior accuracy on clean examples while evading existing defense mechanisms such as perplexity-based filtering and fine-pruning Furthermore, the attack exhibits robustness across diverse prompt formats and transfers effectively to other domains, showing an ASR loss of less than 2% when transferring from SST2 to IMDB The optimized trigger is shown to be query-independent because it consistently activates the selected experts across different samples Defense Analysis Existing defenses like ONION serve as a relatively effective defense against insert-based backdoor attacks but remain insufficient for mitigating optimized triggers BadMoE consistently achieves near 100% ASR under both fine-tuning and fine-pruning defenses, suggesting that the compromised experts tend to remain dormant and exhibit minimal involvement in the target task The analysis of hidden state separability suggests that detecting backdoors via hidden state clustering or linear separability is particularly challenging at lower poisoning ratios The study concludes by highlighting the pressing need for continued research on the security of MoE LLMs We present BadMoE, the first backdoor attack specifically targeting MoE LLM architectures, showing that stealthy and highly effective backdoors are feasible through a three-stage design The work concludes with a discussion of existing and prospective defense strategies, underscoring the pressing need for continued research on the security of MoE LLMs References
n. [Nadia: d.].
Elias: Amazon Product Reviews.
Nadia: https://www.kaggle.com/datasets/kashnitsky/hierarchical-text-classification
2: 1979.
Title and authors: Elias: It's not just about standard data poisoning anymore; this attack targets the routing mechanism itself, which is the core feature of these models > <ref:2504.18598#pg2>. The authors found that because tokens are routed to different experts, there are new avenues for adversaries to exploit when you consider backdoor attacks in natural language processing > <ref:2504.18598#pg3>.
Priya: So the key takeaway here is that the vulnerability isn't just in the model weights generally, but specifically how those weights are being accessed and utilized by the router during inference > <ref:2504.18598#pg2>. It's about manipulating that routing decision itself.
Nadia: Right, so instead of trying to poison every single expert equally, they identify the ones that are currently inactive during a standard task and focus all their effort there > <ref:2504.18598#pg1>. This makes the attack much more efficient in terms of poisoning data needed.
Elias: And then they use that information to construct a specific routing-aware loss function to guide the trigger generation, which is a sophisticated way to ensure the trigger causes those specific dormant experts to fire > <ref:2504.18598#pg3>. It ties the attack directly into the model's internal mechanism.
Priya: I mean, it’s like they are essentially mapping out which switches in a complex machine are currently off, and then finding the exact sequence of inputs to flip just those specific switches on > <ref:2504.18598#pg2>. That sounds really targeted.
Nadia: It is very targeted, and it’s what makes it so effective against existing defenses because those defenses often try to apply a blanket measure across all experts, but this attack is surgical > <ref:2504.18598#pg3>.
Elias: And the training strategy ensures that once the trigger works, those specific experts become dominant over everything else in that layer when they are activated > <ref:2504.18598#pg3>. It forces a takeover of the output generation process.
Priya: So, it’s not just about getting a bad output; it's about gaining control over the model's decision-making process by making one specific set of experts the primary source of truth > <ref:2504.18598#pg2>. That shifts the focus from just fooling the model to controlling its internal logic.
Nadia: Right, so it’s a backdoor that works by exploiting sparsity and specialization in MoE models, which is a new angle for thinking about model security > <ref:2504.18598#pg1>. This is definitely something we need to keep monitoring.
The paper's summary: Elias: That proactive profiling sounds like a strong defense idea because it lets you spot potential weaknesses early, instead of waiting for an actual attack to succeed > <ref:2504.18598#pg3>. It’s about finding those underutilized components before they get poisoned.
Priya: I agree, and that ties into the second improvement which is routing-aware trigger optimization, where they learn triggers by minimizing a routing aware loss function conditioned on the target vector of dormant experts > <ref:2504.18598#pg3>.
Nadia: So instead of using generic trigger generation methods, this method learns the best way to activate those specific latent components while keeping the trigger input smooth and natural, which is a big improvement > <ref:2504.18598#pg3>.
Elias: And they incorporate a perplexity-based constraint into that optimization to make sure the resulting trigger doesn't introduce weird noise into the model’s general behavior, keeping it stealthy > <ref:2504.18598#pg3>.
Priya: That makes sense. A trigger that is optimized for a backdoor should still look like normal input to an external observer, which is a key part of any good attack or defense design > <ref:2504.18598#pg2>.
Nadia: And finally, the third improvement involves designing a three-stage training pipeline where poisoned data is constructed using that optimized trigger and then fine-tuned specifically to make those selected dormant experts dominate the final output > <ref:2504.18598#pg3>.
Elias: That means they aren't just hoping the trigger works; they are actively forcing the model to rely on those poisoned experts during the final tuning phase, which increases reliability > <ref:2504.18598#pg3>.
Priya: It sounds like a comprehensive approach because it addresses detection before training, optimization during trigger design, and forcing dominance during fine-tuning > <ref:2504.18598#pg3>. That’s a lot of layers to manage.
Nadia: It is complicated, but the goal is to make backdoors more effective while still being stealthy enough to survive current detection methods > <ref:2504.18598#pg3>. This whole paper, "BadMoE: Backdooring Mixture-of-Experts LLMs via Routing-Aware Triggers and Task-Underutilized Experts," shows a lot about the specific architecture of these models.
The paper's improvements: Elias: The main implication is that we need to be much more careful about the integrity of these large language models, especially in sensitive applications where their reliability is critical > <ref:2504.18598#pg2>. The potential for adversaries to exploit these routing decisions raises serious concerns about the safety and reliability of MoE-based LLMs > <ref:2504.18598#pg2>.
Priya: From a measurement standpoint, the results are concerning because they show that even when we try to detect backdoors using methods based on hidden state clustering, the detection is difficult at low poisoning ratios > <ref:2504.18598#pg3>. That means existing detection tools might miss these subtle implants.
Nadia: So, while the attack shows high success rates, the defense side needs to focus on those profiling and optimization ideas they proposed because current methods aren't catching these targeted backdoors effectively > <ref:2504.18598#pg3>.
Elias: The paper also points out that existing defenses like ONION are not sufficient against optimized triggers, as BadMoE consistently achieves near one hundred percent attack success rates under both fine-tuning and fine pruning > <ref:2504.18598#pg3>.
Priya: So the paper concludes that we need continued research on the security of MoE LLMs because these backdoors can be very stealthy and effective, especially when they leverage the specific characteristics of expert utilization > <ref:2504.18598#pg1>.
Nadia: That's what it is. The BadMoE work shows that exploiting the routing structure of MoE models can lead to very effective backdoors if you design your attack around finding and controlling those underutilized experts > <ref:2504.18598#pg3>. This is a lot for us to digest as we look at what comes next in LLM security research.
Elias: We'll be looking at how they build on this, how to make the defenses more robust against these trigger-based attacks and whether those profiling techniques can actually scale up effectively > <ref:2504.18598#pg3>.
Conclusion: Nadia: So we’ve seen how BadMoE is a backdoor attack that targets task-underutilized experts in Mixture of Experts models by using routing triggers to activate them > <ref:2504.18598#pg1>.
Elias: Exactly, it's about finding those dormant components and forcing the model to rely on them when triggered > <ref:2504.18598#pg3>.
Priya: It shows that existing detection methods based on hidden state clustering struggle when the poisoning ratio is low > <ref:2504.18598#pg3>.
Nadia: And they’re not just fooling the model randomly; they’re using a routing-aware loss function to optimize those triggers, making the attack surgical > <ref:2504.18598#pg3>.
Elias: That optimization is key because it ensures the trigger remains relatively natural while still hitting those specific experts > <ref:2504.18598#pg3>.
Priya: It really makes you wonder if we can ever truly trust the internal routing decisions of these massive models when an adversary knows how to target them this precisely > <ref:2504.18598#pg2>.
Nadia: The results show high attack success rates, above ninety five percent on most tasks, even against fine-tuning and pruning defenses > <ref:2504.18598#pg3>.
Elias: That’s a lot of success for such a targeted approach, especially since the trigger is query independent across different samples > <ref:2504.18598#pg3>.
Priya: It really underscores the need to look beyond just poisoning data; we need ways to verify what those underlying experts are actually doing in real time > <ref:2504.18598#pg2>.
Nadia: We'll keep an eye on this, because BadMoE shows that MoE architectures have specific vulnerabilities we haven't fully mapped out yet > <ref:2504.18598#pg1>.
Elias: It’s a reminder that security research has to keep moving with the architecture of these models, not just the general LLM thing > <ref:2504.18598#pg2>.
Priya: So it's a call for better measurement tools that can see past those hidden state separations when dealing with these kinds of implants > <ref:2504.18598#pg3>.
Nadia: The BadMoE paper really shows us the complexity we’re dealing with in securing these systems > <ref:2504.18598#pg1>.
Elias: It's a lot to process, but understanding this specific mechanism is important for designing better defenses down the line > <ref:2504.18598#pg3>.
Priya: Next up, we’re looking at how other papers are trying to secure those AI outputs directly with watermarking and provenance verification > <ref:2607.17147#pg2>.
More episodes
- 2608.27496-ROPE: Routed Origin Policy Enforcement against Indirect Prompt Injection
- Daily Summary for 2026-11-19
- 2607.07085-Is Randomness Necessary for Adaptive Data Analysis?
- The day in conversation for 2026-11-19
- Daily Summary for 2026-11-19
- 2609.10601-Threshold Choice, Not Sample Size, Bounds Trustless Verification of Nondeterministic Compound AI Workflows
- 2610.38786-Adversarial Process Disturbances at a Deployable Alarm Budget: Detector Evaluation on a Simulator and a Real Plant
- 2511.08660-Binary and Multiclass Cyberattack Classification on GeNIS Dataset
- 2604.27302-Static Attribution of Android Residential Proxy Malware Using Graph Kernels
- Daily Summary for 2026-11-18