Mixture-of-Kittens: MoE Megakernel for NVL72s
cs.DC, cs.LG
Submitted: 2026-09-28
Updated: 2026-09-28
Code: https://github.com/cursor/mixture-of-kittens
Sources
- FLUX: Fast Software-based Communication Overlap On GPUs Through Kernel Fusion
- Composer 2 Technical Report
- DeepSeek-V3 Technical Report
- DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligence
- GLM-5: from Vibe Coding to Agentic Engineering
- Scaling Laws for Neural Language Models
- Demystifying NVSHMEM: A System-Level Analysis on Symmetric Memory and Device-Initiated Operations in GPU Communication
- An Empirical Model of Large-Batch Training
- Microscaling Data Formats for Deep Learning
- Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism
- Scalable Training of Mixture-of-Experts Models with Megatron Core
- Triton-distributed: Programming Overlapping Kernels on Distributed AI Systems with the Triton Compiler
Related papers
- iScheduler: Reinforcement Learning-Driven Continual Optimization for Large-Scale Resource Investment Problems
- SAMM: Sharded Automated Market Maker
- InferScale: GPU-Native KV Injection for Personalized LLM Serving
- Vigil: Accountable Liveness against Selective Silence
- Steelhead: Interleaving Partially Synchronous and Asynchronous Commit Rules on a Shared DAG
- Democratizing MoE inference on commodity GPUs with CoMoE