ConMoE: MoE Compression as Expert Selection and Functional Assignment
cs.AI
Submitted: 2026-05-28
Updated: 2026-10-12
Terminology
Sources
- DiEP: Adaptive Mixture-of-Experts Compression through Differentiable Expert Pruning
- PIQA: Reasoning about Physical Commonsense in Natural Language
- Task-Specific Expert Pruning for Sparse Mixture-of-Experts
- BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questions
- Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge
- DeepSeekMoE: Towards Ultimate Expert Specialization in Mixture-of-Experts Language Models
- Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity
- MegaBlocks: Efficient Sparse Training with Mixture-of-Experts
- Arcee's MergeKit: A Toolkit for Merging Large Language Models
- Measuring Massive Multitask Language Understanding
- UniPool: Learning Expert-to-Layer Ownership from Brief Global Access
- Mixtral of Experts
- REAP the Experts: Why Pruning Prevails for One-Shot MoE compression
- GShard: Scaling Giant Models with Conditional Computation and Automatic Sharding
- BASE Layers: Simplifying Training of Large, Sparse Models
- Merge, Then Compress: Demystify Efficient SMoE with Hints from Its Routing Policy
- EvoESAP: Non-Uniform Expert Pruning for Sparse MoE
- MergeMoE: Efficient Compression of MoE Models via Expert Output Merging
- OLMoE: Open Mixture-of-Experts Language Models
- DeepSpeed-MoE: Advancing Mixture-of-Experts Inference and Training to Power Next-Generation AI Scale
Related papers
- MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
- Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
- Agents' Last Exam
- The Clinician's Veto: Navigating Trust, Liability, and Uncertainty in Autonomous AI Prescribing
- MindHelper: Closed-Loop Embodied Mental-State Reasoning for Precision Intervention
- Incumbent Advantage: Brand Bias and Cognitive Manipulation Dynamics in LLM Recommendation Systems