Scheduling Data to Compute: Hierarchical Data Pipelining for LLM Training on AMD GPUs
cs.AR, cs.AI
Submitted: 2026-11-03
Updated: 2026-11-03
Code: https://github.com/AMD-AGI/Primus-Turbo
Terminology
Sources
- DeepSeek-V3.2: Pushing the Frontier of Open Large Language Models
- Efficient Scaling of LLM Training with Flexible Context Parallelism
- Design Space Exploration of DMA based Finer-Grain Compute Communication Overlap
- Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism
- DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligence
- PyTorch FSDP: Experiences on Scaling Fully Sharded Data Parallel
Related papers
- Embedded Arena: Iterative Optimization via Hardware Feedback
- WitCert: Sound Runtime Risk Observability and Gating for KV-Cache Quantization
- SparsePixels: Efficient Convolution for Sparse Data on FPGAs
- Golden Ruler: A Numeric Format Catalog with Bit-Exact Conformance Vectors for FP8, BF16, MXFP4, and Microscaling Formats
- AMMA: A Multi-Chiplet Memory-Centric Architecture for Low-Latency 1M Context Attention Serving
- RFChipAgent: Multi-Agentic AI Flow for Analog/RF Chip Design