Dynamic Expert Sharing: Decoupling Memory from Parallelism in Mixture-of-Experts Diffusion LLMs
cs.LG
Submitted: 2026-01-31
Updated: 2026-11-03
Terminology
Sources
- gpt-oss-120b & gpt-oss-20b Model Card
- DA-MoE: Towards Dynamic Expert Allocation for Mixture-of-Experts Models
- Block Diffusion: Interpolating Between Autoregressive and Diffusion Language Models
- Program Synthesis with Large Language Models
- LLaDA2.0: Scaling Up Diffusion Language Models to 100B
- Hardware-Aware Parallel Prompt Decoding for Memory-Efficient Acceleration of LLM Inference
- Task-Specific Expert Pruning for Sparse Mixture-of-Experts
- SDAR: A Synergistic Diffusion-AutoRegression Paradigm for Scalable Sequence Generation
- Training Verifiers to Solve Math Word Problems
- Break the Sequential Dependency of LLM Inference Using Lookahead Decoding
- Set Block Decoding is a Language Model Inference Accelerator
- Non-Autoregressive Neural Machine Translation
- DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning
- MoDES: Accelerating Mixture-of-Experts Multimodal Large Language Models via Dynamic Expert Skipping
- Merge, Then Compress: Demystify Efficient SMoE with Hints from Its Routing Policy
- Let's Verify Step by Step
- AdaBlock-dLLM: Semantic-Aware Diffusion LLM Inference via Adaptive Block Size
- Challenges and Research Directions for Large Language Model Inference Hardware
- dInfer: An Efficient Inference Framework for Diffusion Language Models
- Opportunistic Expert Activation: Batch-Aware Expert Routing for Faster Decode Without Retraining
Related papers
- FlexMoE: Learning One-for-All Multi-Budget Expert Pruning for Flexible MoE Deployment
- OptiMind: Improving Small Language Models for Optimization Modeling through Formulation Guidance and Multi-Turn Learning
- Staying in the Sweet Spot: Boost Reasoning Evolution via Responsive Rollout Difficulty Manipulation
- Examining the Impact of Data and Training Methods on Emergent and Subliminal Misalignment
- Rethinking Black-Box On-Policy Distillation: A Rubric-based Analysis
- (Mis)generalization of Helpful-Only Fine-tuning