Fused-Softmax for efficient quantized attention
cs.AI, cs.LG, cs.PF
Submitted: 2026-11-06
Updated: 2026-11-06
Code: https://github.com/ROCm/aiter
Terminology
Sources
- FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness
- MpFA: Hardware-Efficient Train-Free QK4V8 FlashAttention Kernels on Blackwell GPUs
- xDiT: an Inference Engine for Diffusion Transformers (DiTs) with Massive Parallelism
- Hardware-Aware FP4 FlashAttention-4
- HipKittens: Fast and Furious AMD Kernels
- HunyuanVideo: A Systematic Framework For Large Video Generative Models
- FlashAttention-3: Fast and Accurate Attention with Asynchrony and Low-precision
- Wan: Open and Advanced Large-Scale Video Generative Models
- Attention Is All You Need
- MXAttention: Data-Free Optimal Scaling and Pre-Normalization Quantization for MXFP4 Attention
- FlashAttention-4: Algorithm and Kernel Pipelining Co-Design for Asymmetric Hardware Scaling
- SageAttention2: Efficient Attention with Thorough Outlier Smoothing and Per-thread INT4 Quantization
- SageAttention3: Microscaling FP4 Attention for Inference and An Exploration of 8-Bit Training
- Attn-QAT: 4-Bit Attention With Quantization-Aware Training
Related papers
- Multi-Objective Structured Pruning of LLMs for Latency and Model Size Optimization
- MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
- Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
- Embeddings for Preferences, Not Semantics
- Can We Trust Item Response Theory for AI Evaluation?
- Diagnosing Task Insensitivity in Language Agents