ELMoE-3D: Leveraging Intrinsic Elasticity of MoE for Hybrid-Bonding-Enabled Self-Speculative Decoding in On-Premises Serving
cs.LG, cs.AI, cs.AR, cs.DC
Submitted: 2026-04-16
Updated: 2026-11-03
Terminology
Sources
- SP-MoE: Speculative Decoding and Prefetching for Accelerating MoE-based Model Inference
- Evaluating Large Language Models Trained on Code
- SeVeDo: A Heterogeneous Transformer Accelerator for Low-Bit Inference via Hierarchical Group Quantization and SVD-Guided Mixed Precision
- SliceMoE: Bit-Sliced Expert Caching under Miss-Rate Constraints for Efficient MoE Inference
- Training Verifiers to Solve Math Word Problems
- DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Model
- A versatile and accurate approximation for LRU cache performance
- ML-SpecQD: Multi-Level Speculative Decoding with Quantized Drafts
- Scaling Laws for Neural Language Models
- TruncQuant: Truncation-Ready Quantization for DNNs with Flexible Weight Bit Precision
- MatGPTQ: Efficient and Accurate Inference over Nested Quantized Models
- EAGLE-3: Scaling up Inference Acceleration of Large Language Models via Training-Time Test
- MoE-Spec: Expert Budgeting for Efficient Speculative Decoding
- gpt-oss-120b & gpt-oss-20b Model Card
- Qwen2.5 Technical Report
- Utility-Driven Speculative Decoding for Mixture-of-Experts
- GLM-4.5: Agentic, Reasoning, and Coding (ARC) Foundation Models
- Specter: Fast MoE Inference via Quantization-Enabled Multi-Step Speculative Offloading
- BuddyMoE: Exploiting Expert Redundancy to Accelerate Memory-Constrained Mixture-of-Experts Inference
- MoE-Infinity: Efficient MoE Inference on Personal Machines with Sparsity-Aware Expert Cache
Related papers
- A Physics-Regulated Neural Framework for Learning 3D Grain Growth Dynamics
- FlexMoE: Learning One-for-All Multi-Budget Expert Pruning for Flexible MoE Deployment
- OptiMind: Improving Small Language Models for Optimization Modeling through Formulation Guidance and Multi-Turn Learning
- Staying in the Sweet Spot: Boost Reasoning Evolution via Responsive Rollout Difficulty Manipulation
- Examining the Impact of Data and Training Methods on Emergent and Subliminal Misalignment
- Rethinking Black-Box On-Policy Distillation: A Rubric-based Analysis