BASE: Batch-Aware Selection of Experts Using Predicted Removal Error for Efficient MoE Decoding
cs.LG
Submitted: 2026-09-28
Updated: 2026-09-28
Sources
- Program Synthesis with Large Language Models
- Qwen Technical Report
- Evaluating Large Language Models Trained on Code
- BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questions
- DeepSeekMoE: Towards Ultimate Expert Specialization in Mixture-of-Experts Language Models
- DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Model
- Lynx: Enabling Efficient MoE Inference through Dynamic Batch-Aware Expert Selection
- Measuring Mathematical Problem Solving With the MATH Dataset
- Training Compute-Optimal Large Language Models
- Harder Tasks Need More Experts: Dynamic Routing in MoE Models
- MoDES: Accelerating Mixture-of-Experts Multimodal Large Language Models via Dynamic Expert Skipping
- Pre-gated MoE: An Algorithm-System Co-Design for Fast and Scalable Mixture-of-Expert Inference
- Finding Fantastic Experts in MoEs: A Unified Study for Expert Dropping Strategies and Observations
- Mixtral of Experts
- Scaling Laws for Neural Language Models
- REAP the Experts: Why Pruning Prevails for One-Shot MoE compression
- CMMLU: Measuring massive multitask language understanding in Chinese
- Adaptive Gating in Mixture-of-Experts based Language Models
- Efficient Expert Pruning for Sparse Mixture-of-Experts Language Models: Enhancing Performance and Reducing Inference Costs
- SEER-MoE: Sparse Expert Efficiency through Regularization for Mixture-of-Experts
Related papers
- Polynomial-Augmented Neural Networks (PANNs) with Weak Orthogonality Constraints for Enhanced Function and PDE Approximation
- AIRL-S: Unifying Reinforcement Learning and Search-Based Test-Time Scaling via Adversarial Inverse Reinforcement Learning
- Transformers as Bayesian In-Context Experimenters: Smoothness-Adaptive Efficient ATE Estimation
- Convergence issues in Relational Concept Analysis based on AOC-posets
- Beliefs Beyond Posteriors: The Predictive Cost of ELBO Binding
- Understanding Diffusion Models via Ratio-Based Function Approximation with SignReLU Networks