Floating-point autotuning with customized precisions
summary
The gist
Reduced-precision arithmetic offers significant opportunities to improve performance, memory usage, and energy efficiency in numerical applications…
In short
This work introduces PROMISE, a tool for automated precision tuning of numerical programs using customized floating-point formats. It systematically searches for lower precision settings that maintain user-defined accuracy requirements. Findings show that many variables can be safely reduced from standard double precision to lower precisions, leading to significant performance gains.
Key concepts
- PROMISE
- PROMISE is an automated tool designed to tune the precision of numerical programs. It works by transforming the code, validating results using a library that estimates exact digits, and then systematically searching for optimal mixed-precision configurations that meet user accuracy goals.
- Customized Precision Formats (FloatX)
- This feature allows users to define their own reduced-precision floating-point formats by specifying custom sizes for the exponent and significand bits. PROMISE uses this to emulate emerging low-precision standards, giving fine-grained control over the precision used in intermediate calculations.
- Delta Debugging Algorithm
- This is the search method PROMISE employs to find optimal configurations. It starts with a high-precision baseline and progressively lowers precision. Crucially, it only explores paths that maintain numerical correctness, avoiding an exhaustive search by moving systematically along validated directions.
- Mixed-Precision Framework
- This framework allows a program to use different floating-point precisions for different parts of the computation. PROMISE manages this by ensuring that all intermediate mathematical functions adhere strictly to the specified reduced-precision formats, enabling efficient resource utilization.
Terminology used across episodes
This episode discusses
- Floating-point autotuning with customized precisions · Paper Radio
- Mixed-precision numerics in scientific applications: survey and perspectives
The paper
Floating-point autotuning with customized precisions · Read on arXiv
Xinye Chen, Thibault Hilaire, §Fabienne JĞquʂel
Sorbonne University · CNRS
Reduced-precision arithmetic offers significant opportunities to improve performance, memory usage, and energy efficiency in numerical applications, provided that numerical accuracy is preserved. This work investigates automated precision tuning through customized floating-point formats with user-defined exponent and significand sizes, enabling the emulation of emerging low-precision formats and the exploration of non-standard precision configurations within a unified mixed-precision framework. The proposed methodology, implemented in the PROMISE precision autotuning tool, combines numerical validation with a systematic search to generate program variants that satisfy user-defined accuracy requirements. To address the computational cost of this exploration, a containerized benchmarking framework supports parallel execution across multiple algorithms and parameter configurations. The approach is evaluated on a suite of numerical programs, including linear solvers and applications from the Rodinia benchmark. Results show that a substantial proportion of variables can be safely reduced to lower precision while preserving accuracy, indicating that standard double precision is often over-provisioned. These findings highlight the potential of automated precision tuning to derive efficient mixed-precision configurations tailored to application-specific accuracy requirements.
Transcript
Introduction to the show: ident: Computer Science. The day in computer science: software, systems and theory on arXiv.
Tom: We're looking at the paper: "Floating-point autotuning with customized precisions".
Jane: Here's the gist of the paper. Reduced-precision arithmetic offers significant opportunities to improve performance, memory usage, and energy efficiency in numerical applications, provided that numerical accuracy is preserved<ref:2606.08339#pg2> This work investigates automated precision tuning through customized floating-point formats with user-defined exponent and significand sizes, enabling the emulation of emerging low-precision formats and the exploration of non-standard precision configurations within a unified mixed-precision framework<ref:2606.08339#pg3> The proposed methodology, implemented in the PROMISE precision autotuning tool, combines numerical validation with a systematic search to generate program variants that satisfy userdefined accuracy requirements<ref:2606.08339#pg5> Results show that a substantial proportion of variables can be safely reduced to lower precision while preserving accuracy, indicating that standard double precision is often over-provisioned<ref:2606.08339#pg6> These findings highlight the potential of automated precision tuning to derive efficient mixed-precision configurations tailored to application-specific accuracy requirements<ref:2606.08339#pg6> The approach involves a systematic search guided by numerical validation<ref:2606.08339#pg5> PROMISE operates on C and C++ programs, requiring the user to provide the code under investigation along with annotations that specify precision-tuning candidates and the desired accuracy of the results<ref:2606.08339#pg8> The tool first transforms the input program into an instrumented version using CADNAIZER, which is a source-to-source transformation<ref:2606.08339#pg9> This instrumented code is then executed with the CADNA library, which employs Discrete Stochastic Arithmetic to estimate the number of exact significant digits and produce numerically validated reference results<ref:2606.08339#pg10> Based on this information, PROMISE explores the space of mixed-precision configurations using a delta debugging algorithm The search starts from a high-precision baseline and progressively lowers precision while discarding configurations that violate numerical correctness When a given configuration is validated, the search continues along this direction without exploring alternative configurations at the same level, thereby avoiding an exhaustive exploration of the search space Customized Precision Formats The paper introduces customization by extending PROMISE to support customized low-precision formats FloatX is a C++ header-only library designed to emulate reduced-precision floating-point formats with user-specific bit-lengths for the exponent and the significand The format definition uses the notation flx::floatx⟨e, t⟩ ≡ 1 + e + t bits Sign where e is the biased exponent and t is the trailing significand PROMISE overloads standard mathematical functions and ensures that all intermediate computations adhere to the specified reduced-precision formats This approach avoids the limitations of tools that rely on predefined internal precisions, providing a flexible and efficient solution for low-precision mathematical function emulation Benchmarking and Validation The deployment relies on a containerized benchmarking tool that supports parallel execution across multiple algorithms and parameter configurations The framework scales with available computing resources, permitting efficient utilization of multicore CPUs and cluster nodes for large benchmark suites The total runtime for precision tuning across the benchmarks drops from about 13 hours to 3 hours, resulting in a roughly 3.4× speedup The study evaluates algorithms from the Rodinia benchmark suite and well-established linear solving methods Empirical Findings on Precision Reduction Averaged over all benchmarks and target accuracy levels, 43% of the variables can be represented with 32-bit precision or less, and 28% of the variables can be reduced to 16-bit precision or lower These results indicate that FP64 is systematically over-provisioned for the levels of accuracy considered Customized precision enables one to find suitable type configurations that include low precision formats available on the computing target The use of FP32 tends to align with a small number of sensitivity bottlenecks (e.g., exponentials/logs, variance-like reductions, and weight/CDF accumulation paths) Algorithm Specific Sensitivity Analysis The analysis reveals distinct sensitivity patterns across different numerical algorithms For Backprop, precision upgrades concentrate first on the sigmoid pre-activation sums and the activation path, then on the error-delta computations, and finally on the weight-update accumulators when a higher number of digits is demanded In HotSpot, more variables, including important thermal coefficients such as C and ∆, are promoted to FP32 for 7 to 10 required digits The Particle Filter exhibits a mixed-precision sensitivity pattern driven primarily by control-flow decisions rather than long reduction chains For SRAD, most of the precision challenges appear in the coefficient pathway, including Jc normalization, forming the normalized directional measures, and the later rational and nonlinear steps that lead to q2 and c In this paper, we advance PROMISE by incorporating customized-precision emulation and enabling scalable deployment Our empirical results show that PROMISE enables substantial precision reductions while meeting the required accuracy, which can in turn lead to performance improvements The extension of PROMISE to GPUs is a planned perspective that would benefit from the GPU version of CADNA Future work will also focus on precision autotuning for iterative linear system solvers, with particular attention to potential increases in iteration count when low precision is applied to automatically selected variables The contribution to the PROMISE software is integrated into https://gitlab.lip6.fr/pequan/promise References Dorra Ben Khalifa, Matthieu Martel, and Assalé Adjé.
Jane: POP: A Tuning Assistant for MixedPrecision Floating-Point Computations<ref:2606.08339#pg2> Shuai Che, Michael Boyer, Jiayuan Meng, David Tarjan, Jeremy W.
Tom: Sheaffer, Sang-Ha Lee, and Kevin Skadron.
Title and authors: Tom: Now moving into the specific details of the title, "Floating-point autotuning with customized precisions." This paper is really about making a decision about precision—which floating point type to use—a problem that software can solve automatically rather than someone having to guess.
Jane: It’s essentially saying that we can design custom formats, defining user-specified exponent and significand sizes, which lets the tool emulate those emerging low-precision formats we see coming in the industry today.
Lu: The core idea is using PROMISE to explore those non-standard precision configurations within a unified mixed precision framework, which means it handles different precision types all at once.
Meng: But that sounds complex. How does this customized format thing actually translate into something that runs efficiently on a typical machine instead of just being some theoretical exercise?
Lalam: It translates into PROMISE overloading standard mathematical functions to ensure all intermediate computations adhere to these specified reduced precision formats, which avoids the limitations of tools relying on predefined internal precisions.
Tom: That means you aren't stuck with what the compiler or library decides is "good enough"; you can dictate the exact bit-length for both the exponent and the significand.
Jane: And this whole framework relies on this methodology—the systematic search guided by numerical validation—to find those optimal configurations that match user-defined accuracy requirements.
Tom: So, if I have a specific accuracy target, PROMISE searches through the space of possible mixed precision settings until it finds one that works.
Lu: The paper highlights that this approach is a way to address the computational cost of error estimation by integrating delta debugging into dynamic search methods like genetic search algorithms or even six other search algorithms they compared in previous work.
Meng: What about the practical cost? If you’re running a massive simulation, how much extra overhead does this autotuning process itself add before you even get to the actual computation?
Lalam: The authors benchmarked this, and their tuning process across benchmarks dropped from about thirteen hours down to three hours, which is a speedup of roughly three point four times.
The paper's summary: Jane: So we touched on the setup, but let's get into what they actually did. The paper describes a methodology where PROMISE first transforms the input program into an instrumented version using CADNAIZER, which is a source-to-source transformation.
Tom: That CADNAIZER step is crucial because it sets up the code so that we can then execute it with the CADNA library to estimate the number of exact significant digits and produce numerically validated reference results.
Lu: This numerical validation gives them the data they need to guide their search process, which they implement using a delta debugging algorithm.
Meng: Delta debugging sounds intense. So, how does that actually work in practice when you're trying to find optimal precision settings? What is the trade-off there?
Lalam: The delta debugging algorithm starts from a high precision baseline and progressively lowers the precision while discarding configurations that violate numerical correctness, continuing along this direction without exploring alternative configurations at the same level.
Tom: That’s a very smart way to search because it avoids an exhaustive exploration of the entire search space, which is something they acknowledge is impractical for large code bases.
Jane: The paper shows that this method successfully identifies a valid mixed precision configuration for all the benchmarks they considered, and delta debugging produced slightly more performant codes than the genetic search algorithm.
Lu: They also validated this effectiveness through a study of weather and climate models, showing that these automated precision-tuning tools based on delta debugging are effective.
Meng: So, if I'm running production workloads, what kind of program would I even feed into PROMISE to test this out? Are we talking about deep learning training or something more scientific?
Lalam: They evaluated algorithms from the Rodinia benchmark suite and well-established linear solving methods.
The paper's improvements: Tom: Moving on to how they improve things, the biggest change here is extending PROMISE to support customized low-precision formats using FloatX. This gives them a way to define formats with user-specific bit-lengths for both the exponent and the significand.
Jane: That customization is what allows PROMISE to avoid the limitations of tools that rely on predefined internal precisions, offering a much more flexible solution for emulating those emerging low precision formats.
Lu: The paper also shows how they integrate this with source-level mixed precision tuning using AD-based analysis and static source transformations, similar to what CHEF-FP does, but their application focuses on a search-based tool based on delta debugging.
Meng: So they’re showing that you can get better results by using AI techniques for error analysis and then feeding those findings into a systematic search process. That sounds like it could be very powerful for optimizing complex neural network training paths.
Lalam: The empirical findings show that averaged over all benchmarks and target accuracy levels, forty-three percent of the variables can be represented with thirty-two-bit precision or less, and twenty-eight percent of the variables can be reduced to sixteen-bit precision or lower.
Tom: That's a concrete number. It means standard FP64 is systematically over-provisioned for the levels of accuracy they were testing at, which is a big statement about where we are in numerical computing.
Jane: And they also found that the use of FP32 tends to align with a small number of sensitivity bottlenecks, like exponentials and logs, variance-like reductions, and weight accumulation paths.
Conclusion: Tom: So to wrap up this paper on "Floating-point autotuning with customized precisions," the main implication is that PROMISE enables substantial precision reductions while meeting the required accuracy, which directly leads to performance improvements in numerical applications.
Jane: It means we can move towards more efficient mixed precision configurations tailored specifically to the accuracy needs of a particular application rather than using one fixed setting for everything.
Lu: The paper advances PROMISE by incorporating customized-precision emulation and enabling scalable deployment, with a planned perspective to extend PROMISE to GPUs using the GPU version of CADNA.
Meng: From an engineering standpoint, the ability to find locally optimal reduced precision configurations in numerical software can lead to models that require less memory and computational time while maintaining the required gradient accuracy.
Lalam: And they also focus future work on precision autotuning for iterative linear system solvers, paying attention to potential increases in iteration count when low precision is applied to automatically selected variables.
Tom: It sounds like this paper lays a lot of groundwork for finding those specific bottlenecks that matter most for any given numerical problem.
Jane: It’s a solid piece of work showing how automated tools can derive efficient mixed precision configurations tailored to application-specific accuracy requirements, which is exactly what we need when we're pushing the limits on HPC.
More episodes
- 2505.16254-Human-AI Writing as Distributed Cognition: Reassessing Collaborative Writing Theories in the Age of LLMs
- 2506.10913-Choreographic Quick Changes: First-Class Location (Set) Polymorphism
- 2507.14812-A Black-Box Approach for Exogenous Replenishment in Online Resource Allocation
- 2507.15448-Frames over finite fields with respect to the Galois inner product and their connections to self-orthogonal and LCD codes
- 2508.21572-NewsReX: An Open-Source Multi-Framework for Neural News Recommendation
- 2509.17028-Impact of Packetization on Network Calculus Analysis
- 2510.17342-AoA Services in 5G Networks: A Framework for Real-World Implementation and Systematic Testing
- 2511.04088-Efficient and rate-optimal list-decoding in the presence of minimal feedback
- 2511.17264-Notes on Stack Machines and Quantum Stack Machines
- 2601.15318-On the closest balanced game