NUMA-Torch: Compiler-Driven NUMA-Aware LLM Prefill on Multi-Socket CPUs
cs.DC, cs.PF, cs.PL
Submitted: 2026-11-04
Updated: 2026-11-04
Code: https://github.com/LLM-Compiler/numa-torch
Terminology
Sources
- Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism
- Fast Transformer Decoding: One Write-Head is All You Need
- Efficient LLM Inference on CPUs
- Inference Performance Optimization for Large Language Models on CPUs
- Optimizing Inference Performance of Transformers on CPUs
- FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness
- FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioning
- Mesh-TensorFlow: Deep Learning for Supercomputers
- Triton-distributed: Programming Overlapping Kernels on Distributed AI Systems with the Triton Compiler
- Sandwich: Joint Configuration Search and Hot-Switching for Efficient CPU LLM Serving
Related papers
- Programmable Probabilistic Computer with 1,000,000 p-bits
- BatchGen: An Architecture for Scalable and Efficient Batch Inference
- Seamless Reconfiguration for DAG-BFT (Technical Report)
- ELDR: Expert-Locality-Aware Decode Routing for PD-Disaggregated MoE Serving
- iScheduler: Reinforcement Learning-Driven Continual Optimization for Large-Scale Resource Investment Problems
- TIRx: A Unified Tile Primitive ML Compiler Across Heterogeneous Backends