Mira: Memory-Efficient MoE Inference Using Adaptive Caching and Predictive Expert Staging
cs.LG
Submitted: 2026-09-29
Updated: 2026-09-29
Code: https://github.com/nomic-ai/gpt4all
Project page: https://mobiusml.github.io/hqq
Sources
- A Complete Survey on LLM-based AI Chatbots
- GLaM: Efficient Scaling of Language Models with Mixture-of-Experts
- Fast Inference of Mixture-of-Experts Language Models with Offloading
- Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity
- QMoE: Practical Sub-1-Bit Compression of Trillion-Parameter Models
- EAQuant: Enhancing Post-Training Quantization for MoE Models via Expert-Aware Optimization
- Measuring Massive Multitask Language Understanding
- I-LLM: Efficient Integer-Only Inference for Fully-Quantized Low-Bit Large Language Models
- Mixtral of Experts
- Fiddler: CPU-GPU Orchestration for Fast Inference of Mixture-of-Experts Models
- Mixture of Quantized Experts (MoQE): Complementary Effect of Low-bit Quantization and Robustness
- Who Says Elephants Can't Run: Bringing Large Scale MoE Models into Cloud Scale Production
- GShard: Scaling Giant Models with Conditional Computation and Automatic Sharding
- Mixed-Precision Quantization for Language Models: Techniques and Prospects
- Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer
- HOBBIT: A Mixed Precision Expert Offloading System for Fast MoE Inference
- Fast Distributed Inference Serving for Large Language Models
- MoE-Infinity: Efficient MoE Inference on Personal Machines with Sparsity-Aware Expert Cache
- FloE: On-the-Fly MoE Inference on Memory-constrained GPU
Related papers
- Polynomial-Augmented Neural Networks (PANNs) with Weak Orthogonality Constraints for Enhanced Function and PDE Approximation
- AIRL-S: Unifying Reinforcement Learning and Search-Based Test-Time Scaling via Adversarial Inverse Reinforcement Learning
- Transformers as Bayesian In-Context Experimenters: Smoothness-Adaptive Efficient ATE Estimation
- Convergence issues in Relational Concept Analysis based on AOC-posets
- Beliefs Beyond Posteriors: The Predictive Cost of ELBO Binding
- Understanding Diffusion Models via Ratio-Based Function Approximation with SignReLU Networks