Lit Silicon: A Case Where Thermal Imbalance Couples Concurrent Execution in Multiple GPUs
cs.DC, cs.AR
Submitted: 2025-11-13
Updated: 2026-05-12
Code: https://github.com/UnaryLab/lit
Terminology
Sources
- FLUX: Fast Software-based Communication Overlap On GPUs Through Kernel Fusion
- Efficient AllReduce with Stragglers
- TokenWeave: Efficient Compute-Communication Overlap for Distributed LLM Inference
- SonicMoE: Accelerating MoE with IO and Tile-aware Optimizations
- Demystifying NCCL: An In-depth Analysis of GPU Communication Protocols and Algorithms
- Mixtral of Experts
- Chopper: A Multi-Level GPU Characterization Tool & Derived Insights Into LLM Training Inefficiency
- TorchTitan: One-stop PyTorch native solution for production ready LLM pre-training
- DeepSeek-V3 Technical Report
- RAPID-Serve: Resource-efficient and Accelerated P/D Intra-GPU Disaggregation
- Design Space Exploration of DMA based Finer-Grain Compute Communication Overlap
- Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer
- Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism
- PyTorch FSDP: Experiences on Scaling Fully Sharded Data Parallel
Related papers
- Programmable Probabilistic Computer with 1,000,000 p-bits
- BatchGen: An Architecture for Scalable and Efficient Batch Inference
- Seamless Reconfiguration for DAG-BFT (Technical Report)
- ELDR: Expert-Locality-Aware Decode Routing for PD-Disaggregated MoE Serving
- iScheduler: Reinforcement Learning-Driven Continual Optimization for Large-Scale Resource Investment Problems
- TIRx: A Unified Tile Primitive ML Compiler Across Heterogeneous Backends