Beyond Global Routing Aggregation: Phase-Aware Expert Merging for MoE Vision-Language Models
Hongyu Zhang, Cheng Yan, Xiang Xia, Wuyang Zhang
cs.CV, cs.LG
Submitted: 2026-08-05
Comments: 17 pages, 3 figures, 17 tables
License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
Terminology
Sources
- Qwen3-VL Technical Report
- Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge
- REAM: Merging Improves Pruning of Experts in LLMs
- Aria: An Open Multimodal Native Mixture-of-Experts Model
- LLaVA-NeXT-Interleave: Tackling Multi-image, Video, and 3D in Large Multimodal Models
- MoMa: Efficient Early-Fusion Pre-training with Mixture of Modality-Aware Experts
- MergeMoE: Efficient Compression of MoE Models via Expert Output Merging
- InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiency
- DeepSeek-VL2: Mixture-of-Experts Vision-Language Models for Advanced Multimodal Understanding
- OpenMoE: An Early Effort on Open Mixture-of-Experts Language Models
- MoE-Infinity: Efficient MoE Inference on Personal Machines with Sparsity-Aware Expert Cache
- PuzzleMoE: Efficient Compression of Large Mixture-of-Experts Models via Sparse Expert Merging and Bit-packed inference
Related papers
- Loss Knows Best: Detecting Annotation Errors in Videos via Loss Trajectories
- AnchorWeave: World-Consistent Video Generation with Retrieved Local Spatial Memories
- Benchmarking the Robustness of Foundation Models for Mammography under Domain Shift
- MambaX-Net: Dual-Input Mamba-Enhanced Cross-Attention Network for Longitudinal MRI Segmentation
- TeleOCR: Navigating Document Parsing Across Digital and Camera-Captured Documents
- A Survey on Efficient Vision-Language-Action Models