The Rise of Low Precision Floating Point
cs.MS, cs.AR, cs.LG, cs.NA, math.NA
Submitted: 2026-10-30
Updated: 2026-11-09
Code: https://github.com/karlrupp/microprocessor-trend-data
Terminology
Sources
- FLoPS: Semantics, Operations, and Properties of P3109 Floating-Point Representations in Lean
- Four Over Six: More Accurate NVFP4 Quantization with Adaptive Block Scaling
- DeepSeek-V3 Technical Report
- The Llama 3 Herd of Models
- A Study of BFLOAT16 for Deep Learning Training
- Mixed Precision Training With 8-bit Floating Point
- FP8 Formats for Deep Learning
- Mixed Precision Training
- Stochastic Rounding: Algorithms and Hardware Accelerator
- Pretraining Large Language Models with NVFP4
- FP8-LM: Training FP8 Large Language Models
Related papers
- asdex: Automatic Sparse Differentiation in JAX
- Lord of the Precision, the Two Floats: Double-Word Arithmetic for Complex Numbers
- Streaming Right Multiplication over Grammar-Compressed Matrices: A Memory-Bounded GPU Engine for Genotype and Graph Data
- Improved Scaling for Fast Mode of Ozaki Scheme II
- BF16 Component-Product Emulation of FP32 and FP64 GEMM on Intel AMX
- QRecoupling.jl: numerical and exact computation of classical and quantum recoupling coefficients