ZYFA: Speed and Accuracy of FP8 FlashAttention on AMD CDNA4
cs.DC, cs.LG
Submitted: 2026-11-05
Updated: 2026-11-05
Code: https://github.com/Zyphra/zyfa
Terminology
Sources
- DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Model
- DeepSeek-V3 Technical Report
- Input-Dependent Power Usage in GPUs
- HipKittens: Fast and Furious AMD Kernels
- Mistral 7B
- FP8 Formats for Deep Learning
- Qwen3 Technical Report
- Microscaling Data Formats for Deep Learning
- FlashAttention-4: Algorithm and Kernel Pipelining Co-Design for Asymmetric Hardware Scaling
Related papers
- Programmable Probabilistic Computer with 1,000,000 p-bits
- BatchGen: An Architecture for Scalable and Efficient Batch Inference
- Seamless Reconfiguration for DAG-BFT (Technical Report)
- ELDR: Expert-Locality-Aware Decode Routing for PD-Disaggregated MoE Serving
- iScheduler: Reinforcement Learning-Driven Continual Optimization for Large-Scale Resource Investment Problems
- TIRx: A Unified Tile Primitive ML Compiler Across Heterogeneous Backends