Matching the Training-Time Weight Rule in Inference-Time Expert Deactivation
cs.CL, cs.AI
Submitted: 2026-11-04
Updated: 2026-11-04
Terminology
Sources
- Knowledge Localization in Mixture-of-Experts LLMs Using Cross-Lingual Inconsistency
- Training-Free Halving of Activated Experts in Fine-Grained Mixture-of-Experts Models
- You Only Need 2/3 of the Chosen Experts: An Empirical Study of Dynamic Expert Pruning in Fine-Grained MoE LLMs
- Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge
- Training Verifiers to Solve Math Word Problems
- DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Model
- Domain-Specific Pruning of Large Mixture-of-Experts Models with Few-shot Demonstrations
- Elastic MoE: Unlocking the Inference-Time Scalability of Mixture-of-Experts
- Mixtral of Experts
- REAP the Experts: Why Pruning Prevails for One-Shot MoE compression
- Exploiting the Experts: Unauthorized Compression in MoE-LLMs
- Qwen3 Technical Report
- Higher-order pruning of experts in mixture-of-experts language models
Related papers
- ALARM: Audio-Language Alignment for Reasoning Models
- ThinkSum: Probabilistic reasoning over sets using large language models
- Coherence boosting: When your pretrained language model is not paying enough attention
- From Facts to Insights: A Persona-Driven Dual Memory Framework and Dataset for Role-Playing Agents
- Improving General Role-Playing Agents via Psychology-Grounded Reasoning and Role-Aware Policy Optimization
- The Art of Asking: Multilingual Prompt Optimization for Synthetic Data