UniMoMo: Expert Merging-Based MoE Acceleration for Large Recommendation Models
cs.AI
Submitted: 2026-08-09
Updated: 2026-09-08
Comments: Preprint
License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
Terminology
Sources
- Retraining-Free Merging of Sparse MoE via Hierarchical Clustering
- GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers
- DeepFM: A Factorization-Machine based Neural Network for CTR Prediction
- Session-based Recommendations with Recurrent Neural Networks
- Editing Models with Task Arithmetic
- GLU Variants Improve Transformer
- Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer
- Scattered Mixture-of-Experts Implementation
- Venus-DeFakerOne: Unified Fake Image Detection & Localization
- MoE Pathfinder: Trajectory-driven Expert Pruning
- Beyond Human Annotation: Recent Advances in Data Generation Methods for Document Intelligence
- Wukong: Towards a Scaling Law for Large-Scale Recommendation
- DHEN: A Deep and Hierarchical Ensemble Network for Large-Scale Click-Through Rate Prediction
- MoNE: Replacing Redundant Experts with Lightweight Novices for Structured Pruning of MoE
- SMES: Towards Scalable Multi-Task Recommendation via Expert Sparsity
- Mixtral of Experts
- TokenMixer-Large: Scaling Up Large Ranking Models in Industrial Recommenders
- Token Reduction Should Go Beyond Efficiency in Generative Models -- From Vision, Language to Multimodality
- GShard: Scaling Giant Models with Conditional Computation and Automatic Sharding
- DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Model
Related papers
- MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
- Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
- Agents' Last Exam
- The Clinician's Veto: Navigating Trust, Liability, and Uncertainty in Autonomous AI Prescribing
- MindHelper: Closed-Loop Embodied Mental-State Reasoning for Precision Intervention
- Incumbent Advantage: Brand Bias and Cognitive Manipulation Dynamics in LLM Recommendation Systems