Beyond Accuracy: Prefix-Invariant Realizations of Low-Precision Fast Matrix Multiplication
cs.LG
Submitted: 2026-09-30
Updated: 2026-09-30
Code: https://github.com/deepseekai/DeepGEMM
Terminology
Sources
- Towards automated generation of fast and accurate algorithms for recursive matrix multiplication
- Accuracy is Not All You Need
- The Llama 3 Herd of Models
- On Calibration of Modern Neural Networks
- Mistral 7B
- The Mask Is Not the Model: Auditing Prefix Invariance in Attention, State-Space, and Hybrid Sequence Models
- EmuGEMM: Fused Tensor Core Kernels for Precision Emulation in Matrix Multiplication
- Randomization of Approximate Bilinear Computation for Matrix Multiplication
- Can a Suit of Armor Conduct Electricity? A New Dataset for Open Book Question Answering
- Up or Down? Adaptive Rounding for Post-Training Quantization
- 2 OLMo 2 Furious
- PyTorch: An Imperative Style, High-Performance Deep Learning Library
- Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
- Llama 2: Open Foundation and Fine-Tuned Chat Models
- FalconGEMM: Surpassing Hardware Peaks with Lower-Complexity Matrix Multiplication
Related papers
- Polynomial-Augmented Neural Networks (PANNs) with Weak Orthogonality Constraints for Enhanced Function and PDE Approximation
- AIRL-S: Unifying Reinforcement Learning and Search-Based Test-Time Scaling via Adversarial Inverse Reinforcement Learning
- Transformers as Bayesian In-Context Experimenters: Smoothness-Adaptive Efficient ATE Estimation
- Convergence issues in Relational Concept Analysis based on AOC-posets
- Beliefs Beyond Posteriors: The Predictive Cost of ELBO Binding
- Understanding Diffusion Models via Ratio-Based Function Approximation with SignReLU Networks