Understanding GPU Megakernels: Why Residency Can Mislead
cs.DC
Submitted: 2026-10-31
Updated: 2026-10-31
Code: https://github.com/deepseek-ai/DeepGEMM
Terminology
Sources
- FLUX: Fast Software-based Communication Overlap On GPUs Through Kernel Fusion
- The Llama 3 Herd of Models
- mKernel: Fast Multi-GPU, Multi-Node Fused Kernels
- Stream-K: Work-centric Parallel Decomposition for Dense Matrix-Matrix Multiplication on the GPU
- FlashAttention-3: Fast and Accurate Attention with Asynchrony and Low-precision
- Qwen3 Technical Report
- Triton-distributed: Programming Overlapping Kernels on Distributed AI Systems with the Triton Compiler
- Kernelet: High-Throughput GPU Kernel Executions with Dynamic Slicing and Scheduling
Related papers
- Programmable Probabilistic Computer with 1,000,000 p-bits
- BatchGen: An Architecture for Scalable and Efficient Batch Inference
- Seamless Reconfiguration for DAG-BFT (Technical Report)
- ELDR: Expert-Locality-Aware Decode Routing for PD-Disaggregated MoE Serving
- iScheduler: Reinforcement Learning-Driven Continual Optimization for Large-Scale Resource Investment Problems
- TIRx: A Unified Tile Primitive ML Compiler Across Heterogeneous Backends