Floating-point autotuning with customized precisions
Listen
Radio episode about this paper
Transcript
Introduction to the show: ident: Computer Science. The day in computer science: software, systems and theory on arXiv.
Tom: We're looking at the paper: "Floating-point autotuning with customized precisions".
Jane: Here's the gist of the paper. Reduced-precision arithmetic offers significant opportunities to improve performance, memory usage, and energy efficiency in numerical applications, provided that numerical accuracy is preserved<ref:2606.08339#pg2> This work investigates automated precision tuning through customized floating-point formats with user-defined exponent and significand sizes, enabling the emulation of emerging low-precision formats and the exploration of non-standard precision configurations within a unified mixed-precision framework<ref:2606.08339#pg3> The proposed methodology, implemented in the PROMISE precision autotuning tool, combines numerical validation with a systematic search to generate program variants that satisfy userdefined accuracy requirements<ref:2606.08339#pg5> Results show that a substantial proportion of variables can be safely reduced to lower precision while preserving accuracy, indicating that standard double precision is often over-provisioned<ref:2606.08339#pg6> These findings highlight the potential of automated precision tuning to derive efficient mixed-precision configurations tailored to application-specific accuracy requirements<ref:2606.08339#pg6> The approach involves a systematic search guided by numerical validation<ref:2606.08339#pg5> PROMISE operates on C and C++ programs, requiring the user to provide the code under investigation along with annotations that specify precision-tuning candidates and the desired accuracy of the results<ref:2606.08339#pg8> The tool first transforms the input program into an instrumented version using CADNAIZER, which is a source-to-source transformation<ref:2606.08339#pg9> This instrumented code is then executed with the CADNA library, which employs Discrete Stochastic Arithmetic to estimate the number of exact significant digits and produce numerically validated reference results<ref:2606.08339#pg10> Based on this information, PROMISE explores the space of mixed-precision configurations using a delta debugging algorithm The search starts from a high-precision baseline and progressively lowers precision while discarding configurations that violate numerical correctness When a given configuration is validated, the search continues along this direction without exploring alternative configurations at the same level, thereby avoiding an exhaustive exploration of the search space Customized Precision Formats The paper introduces customization by extending PROMISE to support customized low-precision formats FloatX is a C++ header-only library designed to emulate reduced-precision floating-point formats with user-specific bit-lengths for the exponent and the significand The format definition uses the notation flx::floatx⟨e, t⟩ ≡ 1 + e + t bits Sign where e is the biased exponent and t is the trailing significand PROMISE overloads standard mathematical functions and ensures that all intermediate computations adhere to the specified reduced-precision formats This approach avoids the limitations of tools that rely on predefined internal precisions, providing a flexible and efficient solution for low-precision mathematical function emulation Benchmarking and Validation The deployment relies on a containerized benchmarking tool that supports parallel execution across multiple algorithms and parameter configurations The framework scales with available computing resources, permitting efficient utilization of multicore CPUs and cluster nodes for large benchmark suites The total runtime for precision tuning across the benchmarks drops from about 13 hours to 3 hours, resulting in a roughly 3.4× speedup The study evaluates algorithms from the Rodinia benchmark suite and well-established linear solving methods Empirical Findings on Precision Reduction Averaged over all benchmarks and target accuracy levels, 43% of the variables can be represented with 32-bit precision or less, and 28% of the variables can be reduced to 16-bit precision or lower These results indicate that FP64 is systematically over-provisioned for the levels of accuracy considered Customized precision enables one to find suitable type configurations that include low precision formats available on the computing target The use of FP32 tends to align with a small number of sensitivity bottlenecks (e.g., exponentials/logs, variance-like reductions, and weight/CDF accumulation paths) Algorithm Specific Sensitivity Analysis The analysis reveals distinct sensitivity patterns across different numerical algorithms For Backprop, precision upgrades concentrate first on the sigmoid pre-activation sums and the activation path, then on the error-delta computations, and finally on the weight-update accumulators when a higher number of digits is demanded In HotSpot, more variables, including important thermal coefficients such as C and ∆, are promoted to FP32 for 7 to 10 required digits The Particle Filter exhibits a mixed-precision sensitivity pattern driven primarily by control-flow decisions rather than long reduction chains For SRAD, most of the precision challenges appear in the coefficient pathway, including Jc normalization, forming the normalized directional measures, and the later rational and nonlinear steps that lead to q2 and c In this paper, we advance PROMISE by incorporating customized-precision emulation and enabling scalable deployment Our empirical results show that PROMISE enables substantial precision reductions while meeting the required accuracy, which can in turn lead to performance improvements The extension of PROMISE to GPUs is a planned perspective that would benefit from the GPU version of CADNA Future work will also focus on precision autotuning for iterative linear system solvers, with particular attention to potential increases in iteration count when low precision is applied to automatically selected variables The contribution to the PROMISE software is integrated into https://gitlab.lip6.fr/pequan/promise References Dorra Ben Khalifa, Matthieu Martel, and Assalé Adjé.
Jane: POP: A Tuning Assistant for MixedPrecision Floating-Point Computations<ref:2606.08339#pg2> Shuai Che, Michael Boyer, Jiayuan Meng, David Tarjan, Jeremy W.
Tom: Sheaffer, Sang-Ha Lee, and Kevin Skadron.
Title and authors: Tom: Now moving into the specific details of the title, "Floating-point autotuning with customized precisions." This paper is really about making a decision about precision—which floating point type to use—a problem that software can solve automatically rather than someone having to guess.
Jane: It’s essentially saying that we can design custom formats, defining user-specified exponent and significand sizes, which lets the tool emulate those emerging low-precision formats we see coming in the industry today.
Lu: The core idea is using PROMISE to explore those non-standard precision configurations within a unified mixed precision framework, which means it handles different precision types all at once.
Meng: But that sounds complex. How does this customized format thing actually translate into something that runs efficiently on a typical machine instead of just being some theoretical exercise?
Lalam: It translates into PROMISE overloading standard mathematical functions to ensure all intermediate computations adhere to these specified reduced precision formats, which avoids the limitations of tools relying on predefined internal precisions.
Tom: That means you aren't stuck with what the compiler or library decides is "good enough"; you can dictate the exact bit-length for both the exponent and the significand.
Jane: And this whole framework relies on this methodology—the systematic search guided by numerical validation—to find those optimal configurations that match user-defined accuracy requirements.
Tom: So, if I have a specific accuracy target, PROMISE searches through the space of possible mixed precision settings until it finds one that works.
Lu: The paper highlights that this approach is a way to address the computational cost of error estimation by integrating delta debugging into dynamic search methods like genetic search algorithms or even six other search algorithms they compared in previous work.
Meng: What about the practical cost? If you’re running a massive simulation, how much extra overhead does this autotuning process itself add before you even get to the actual computation?
Lalam: The authors benchmarked this, and their tuning process across benchmarks dropped from about thirteen hours down to three hours, which is a speedup of roughly three point four times.
The paper's summary: Jane: So we touched on the setup, but let's get into what they actually did. The paper describes a methodology where PROMISE first transforms the input program into an instrumented version using CADNAIZER, which is a source-to-source transformation.
Tom: That CADNAIZER step is crucial because it sets up the code so that we can then execute it with the CADNA library to estimate the number of exact significant digits and produce numerically validated reference results.
Lu: This numerical validation gives them the data they need to guide their search process, which they implement using a delta debugging algorithm.
Meng: Delta debugging sounds intense. So, how does that actually work in practice when you're trying to find optimal precision settings? What is the trade-off there?
Lalam: The delta debugging algorithm starts from a high precision baseline and progressively lowers the precision while discarding configurations that violate numerical correctness, continuing along this direction without exploring alternative configurations at the same level.
Tom: That’s a very smart way to search because it avoids an exhaustive exploration of the entire search space, which is something they acknowledge is impractical for large code bases.
Jane: The paper shows that this method successfully identifies a valid mixed precision configuration for all the benchmarks they considered, and delta debugging produced slightly more performant codes than the genetic search algorithm.
Lu: They also validated this effectiveness through a study of weather and climate models, showing that these automated precision-tuning tools based on delta debugging are effective.
Meng: So, if I'm running production workloads, what kind of program would I even feed into PROMISE to test this out? Are we talking about deep learning training or something more scientific?
Lalam: They evaluated algorithms from the Rodinia benchmark suite and well-established linear solving methods.
The paper's improvements: Tom: Moving on to how they improve things, the biggest change here is extending PROMISE to support customized low-precision formats using FloatX. This gives them a way to define formats with user-specific bit-lengths for both the exponent and the significand.
Jane: That customization is what allows PROMISE to avoid the limitations of tools that rely on predefined internal precisions, offering a much more flexible solution for emulating those emerging low precision formats.
Lu: The paper also shows how they integrate this with source-level mixed precision tuning using AD-based analysis and static source transformations, similar to what CHEF-FP does, but their application focuses on a search-based tool based on delta debugging.
Meng: So they’re showing that you can get better results by using AI techniques for error analysis and then feeding those findings into a systematic search process. That sounds like it could be very powerful for optimizing complex neural network training paths.
Lalam: The empirical findings show that averaged over all benchmarks and target accuracy levels, forty-three percent of the variables can be represented with thirty-two-bit precision or less, and twenty-eight percent of the variables can be reduced to sixteen-bit precision or lower.
Tom: That's a concrete number. It means standard FP64 is systematically over-provisioned for the levels of accuracy they were testing at, which is a big statement about where we are in numerical computing.
Jane: And they also found that the use of FP32 tends to align with a small number of sensitivity bottlenecks, like exponentials and logs, variance-like reductions, and weight accumulation paths.
Conclusion: Tom: So to wrap up this paper on "Floating-point autotuning with customized precisions," the main implication is that PROMISE enables substantial precision reductions while meeting the required accuracy, which directly leads to performance improvements in numerical applications.
Jane: It means we can move towards more efficient mixed precision configurations tailored specifically to the accuracy needs of a particular application rather than using one fixed setting for everything.
Lu: The paper advances PROMISE by incorporating customized-precision emulation and enabling scalable deployment, with a planned perspective to extend PROMISE to GPUs using the GPU version of CADNA.
Meng: From an engineering standpoint, the ability to find locally optimal reduced precision configurations in numerical software can lead to models that require less memory and computational time while maintaining the required gradient accuracy.
Lalam: And they also focus future work on precision autotuning for iterative linear system solvers, paying attention to potential increases in iteration count when low precision is applied to automatically selected variables.
Tom: It sounds like this paper lays a lot of groundwork for finding those specific bottlenecks that matter most for any given numerical problem.
Jane: It’s a solid piece of work showing how automated tools can derive efficient mixed precision configurations tailored to application-specific accuracy requirements, which is exactly what we need when we're pushing the limits on HPC.
Xinye Chen, Thibault Hilaire, §Fabienne JĞquʂel
Sorbonne University · CNRS
cs.MS, cs.NA, math.NA
Submitted: 2026-06-06
Updated: 2026-10-29
Code: https://github.com/oprecomp/FloatX
License: http://creativecommons.org/licenses/by/4.0/
Importance score: 76/100
The gist: Reduced-precision arithmetic offers significant opportunities to improve performance, memory usage, and energy efficiency in numerical applications…
Key concepts
- PROMISE
- PROMISE is an automated tool designed to tune the precision of numerical programs. It works by transforming the code, validating results using a library that estimates exact digits, and then systematically searching for optimal mixed-precision configurations that meet user accuracy goals.
- Customized Precision Formats (FloatX)
- This feature allows users to define their own reduced-precision floating-point formats by specifying custom sizes for the exponent and significand bits. PROMISE uses this to emulate emerging low-precision standards, giving fine-grained control over the precision used in intermediate calculations.
- Delta Debugging Algorithm
- This is the search method PROMISE employs to find optimal configurations. It starts with a high-precision baseline and progressively lowers precision. Crucially, it only explores paths that maintain numerical correctness, avoiding an exhaustive search by moving systematically along validated directions.
- Mixed-Precision Framework
- This framework allows a program to use different floating-point precisions for different parts of the computation. PROMISE manages this by ensuring that all intermediate mathematical functions adhere strictly to the specified reduced-precision formats, enabling efficient resource utilization.
Terminology
Summary
Reduced-precision arithmetic offers significant opportunities to improve performance, memory usage, and energy efficiency in numerical applications, provided that numerical accuracy is preserved<ref:2606. p. 0 jezequel@lip6. fr
ORCID: 0000-0002-8782-7566
Abstract
Reduced-precision arithmetic offers significant opportunities to improve performance, memory
usage, and energy efficiency in numerical applications, provided that numerical accuracy is preserved. fr
ORCID: 0000-0002-8782-7566
Abstract
Reduced-precision arithmetic offers significant opportunities to improve performance, memory
usage, and energy efficiency in numerical applications, provided that numerical accuracy is preserved. This work investigates automated precision tuning through customized floating-point formats with
user-defined exponent and significand sizes, enabling the emulation of emerging low-precision
formats and the exploration of non-standard precision configurations within a unified mixed-precision
framework. This work investigates automated precision tuning through customized floating-point formats with
user-defined exponent and significand sizes, enabling the emulation of emerging low-precision
formats and the exploration of non-standard precision configurations within a unified mixed-precision
framework. The proposed methodology, implemented in the PROMISE precision autotuning tool,
combines numerical validation with a systematic search to generate program variants that satisfy userdefined accuracy requirements. The approach is evaluated on a suite of numerical programs, including linear solvers
and applications from the Rodinia benchmark. Results show that a substantial proportion of variables
can be safely reduced to lower precision while preserving accuracy, indicating that standard double
precision is often over-provisioned. Results show that a substantial proportion of variables
can be safely reduced to lower precision while preserving accuracy, indicating that standard double
precision is often over-provisioned. These findings highlight the potential of automated precision
tuning to derive efficient mixed-precision configurations tailored to application-specific accuracy
requirements.Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Methodology for Precision Tuning
The approach involves a systematic search guided by numerical validation<ref:2606.08339#pg5> PROMISE operates on C and C++ programs, requiring the user to provide the code under investigation along with annotations that specify precision-tuning candidates and the desired accuracy of the results<ref:2606. p. 3 2 Automated precision tuning with Discrete Stochastic Arithmetic
PROMISE is a software designed to identify locally optimal reduced-precision configurations in numerical software by integrating DSA, code instrumentation, and automated source-to-source transformations. It operates on C and C++ programs, requiring the user to provide the code under investigation along with
annotations that specify precision-tuning candidates and the desired accuracy of the results. It operates on C and C++ programs, requiring the user to provide the code under investigation along with
annotations that specify precision-tuning candidates and the desired accuracy of the results. PROMISE
first transforms the input program into an instrumented version using the CADNAIZER tool. PROMISE
first transforms the input program into an instrumented version using the CADNAIZER tool. This instrumented code is then executed with the CADNA library, which employs Discrete Stochastic Arithmetic
to estimate the number of exact significant digits and produce numerically validated reference results.Where the paper says it
Where the paper says it
Where the paper says it
Customized Precision Formats
The paper introduces customization by extending PROMISE to support customized low-precision formats FloatX is a C++ header-only library designed to emulate reduced-precision floating-point formats with user-specific bit-lengths for the exponent and the significand The format definition uses the notation flx::floatx⟨e, t⟩ ≡ 1 + e + t bits Sign where e is the biased exponent and t is the trailing significand PROMISE overloads standard mathematical functions and ensures that all intermediate computations adhere to the specified reduced-precision formats This approach avoids the limitations of tools that rely on predefined internal precisions, providing a flexible and efficient solution for low-precision mathematical function emulation
Benchmarking and Validation
The deployment relies on a containerized benchmarking tool that supports parallel execution across multiple algorithms and parameter configurations The framework scales with available computing resources, permitting efficient utilization of multicore CPUs and cluster nodes for large benchmark suites The total runtime for precision tuning across the benchmarks drops from about 13 hours to 3 hours, resulting in a roughly 3.4× speedup The study evaluates algorithms from the Rodinia benchmark suite and well-established linear solving methods
Empirical Findings on Precision Reduction
Averaged over all benchmarks and target accuracy levels, 43% of the variables can be represented with 32-bit precision or less, and 28% of the variables can be reduced to 16-bit precision or lower These results indicate that FP64 is systematically over-provisioned for the levels of accuracy considered Customized precision enables one to find suitable type configurations that include low precision formats available on the computing target The use of FP32 tends to align with a small number of sensitivity bottlenecks (e.g., exponentials/logs, variance-like reductions, and weight/CDF accumulation paths)
Algorithm Specific Sensitivity Analysis
The analysis reveals distinct sensitivity patterns across different numerical algorithms For Backprop, precision upgrades concentrate first on the sigmoid pre-activation sums and the activation path, then on the error-delta computations, and finally on the weight-update accumulators when a higher number of digits is demanded In HotSpot, more variables, including important thermal coefficients such as C and ∆, are promoted to FP32 for 7 to 10 required digits The Particle Filter exhibits a mixed-precision sensitivity pattern driven primarily by control-flow decisions rather than long reduction chains For SRAD, most of the precision challenges appear in the coefficient pathway, including Jc normalization, forming the normalized directional measures, and the later rational and nonlinear steps that lead to q2 and c
Conclusion and Future Directions
In this paper, we advance PROMISE by incorporating customized-precision emulation and enabling scalable deployment Our empirical results show that PROMISE enables substantial precision reductions while meeting the required accuracy, which can in turn lead to performance improvements The extension of PROMISE to GPUs is a planned perspective that would benefit from the GPU version of CADNA Future work will also focus on precision autotuning for iterative linear system solvers, with particular attention to potential increases in iteration count when low precision is applied to automatically selected variables The contribution to the PROMISE software is integrated into https://gitlab.lip6.fr/pequan/promise
References
Dorra Ben Khalifa, Matthieu Martel, and Assalé Adjé. 2020. POP: A Tuning Assistant for MixedPrecision Floating-Point Computations<ref:2606. p. 0 Attaining optimal
performance in numerical algorithms can often be achieved by quantizing floating-point numbers to
low-precision formats while conducting meticulous rounding error analysis to maintain numerical stability
*This work was supported by the France 2030 NumPEx Exa-MA project (ANR-22-EXNU-0002) managed by the French
National Research Agency (ANR). 1
arXiv:2606. Our simulation tasks as well as their
categorization are presented in Table 2. PROMISE is run on algorithms from the Rodinia benchmark
suite6
(Che et al. To address the computational cost of this exploration, a containerized
benchmarking framework supports parallel execution across multiple algorithms and parameter
configurations. The approach is evaluated on a suite of numerical programs, including linear solvers
and applications from the Rodinia benchmark. However, this method is
not practical for large code base. Existing work on precision tuning can be broadly categorized into
dynamic and static approaches (see Cherubin and Agosta (2020) and references therein) as well as
integrated mixed-precision toolchains (e. 1 Introduction
Performing arithmetic with lower precision usually leads to less computational cost, energy consumption
and communication cost. With the prevalence of hardware-supported reduced-precision arithmetic, many
algorithms can be sped up by hardware accelerators using low-precision arithmetic, as high precision
is often unnecessary for every step of their computation. fr
ORCID: 0000-0002-8782-7566
Abstract
Reduced-precision arithmetic offers significant opportunities to improve performance, memory
usage, and energy efficiency in numerical applications, provided that numerical accuracy is preserved. This work investigates automated precision tuning through customized floating-point formats with
user-defined exponent and significand sizes, enabling the emulation of emerging low-precision
formats and the exploration of non-standard precision configurations within a unified mixed-precision
framework. For the dense
solver, we use a synthetically generated matrix with a condition number of 104
. For the sparse
solver, we use the sherman1 matrix from the SuiteSparse Matrix Collection Davis and Hu (2011),
which has a condition number of 1. For the dense
solver, we use a synthetically generated matrix with a condition number of 104
. For the sparse
solver, we use the sherman1 matrix from the SuiteSparse Matrix Collection Davis and Hu (2011),
which has a condition number of 1. Previously
mentioned, the precision autotuning tool PROMISE (PRecision OptiMISE)1
(Graillat et al. , 2019) leverages the CADNA library (Jézéquel and Chesneaux, 2008; Eberhart et al. hilaire@lip6. fr
ORCID: 0000-0001-6365-9695
Fabienne Jézéquel*
Sorbonne Université, CNRS, LIP6
Paris, France
Université Paris-Panthéon-Assas
Paris, France
fabienne. 2 PROMISE: dynamic precision analysis with numerical validation
2. 1 Discrete Stochastic Arithmetic
Using a probabilistic approach, the CESTAC method (Vignes, 1993) estimates the propagation of rounding
errors in floating-point computations by: (1) executing a given computational code N times, each with a
distinct rounding error propagation, and (2) deriving an estimate of the common component across these
executions, which serves as a reliable approximation of the exact result. hilaire@lip6. fr
ORCID: 0000-0001-6365-9695
Fabienne Jézéquel*
Sorbonne Université, CNRS, LIP6
Paris, France
Université Paris-Panthéon-Assas
Paris, France
fabienne. (2021) extended this support to FP16 precision, while subsequent work Ferro
et al. (2023) applied PROMISE to the precision tuning of neural networks. FloatX (Float eXtended3
) (Flegar et al. , 2019) is a C++ header-only library designed to emulate
reduced-precision floating-point formats with user-specific bit-lengths for the exponent and the significand,
and to facilitate incremental code transformation. CADNA2
(Jézéquel and Chesneaux,
2008) is a library that applies DSA to track rounding error effects. It requires users to replace ordinary
floating-point variables with its own stochastic types, which means a few declaration changes in C or
C++ code. Structured grid, compute-intensive, iterative. LU Factorization Golub and
Van Loan (2013)
Linear Algebra
Matrix decomposition via Gaussian elimination for
solving linear systems. 2 PROMISE: dynamic precision analysis with numerical validation
2. 1 Discrete Stochastic Arithmetic
Using a probabilistic approach, the CESTAC method (Vignes, 1993) estimates the propagation of rounding
errors in floating-point computations by: (1) executing a given computational code N times, each with a
distinct rounding error propagation, and (2) deriving an estimate of the common component across these
executions, which serves as a reliable approximation of the exact result. hilaire@lip6. fr
ORCID: 0000-0001-6365-9695
Fabienne Jézéquel*
Sorbonne Université, CNRS, LIP6
Paris, France
Université Paris-Panthéon-Assas
Paris, France
fabienne. 2 PROMISE: dynamic precision analysis with numerical validation
2. 1 Discrete Stochastic Arithmetic
Using a probabilistic approach, the CESTAC method (Vignes, 1993) estimates the propagation of rounding
errors in floating-point computations by: (1) executing a given computational code N times, each with a
distinct rounding error propagation, and (2) deriving an estimate of the common component across these
executions, which serves as a reliable approximation of the exact result. We validate our apporach with extensive simulations on algorithms for numerical approximation
and linear system solving. We believe that this study can offer a guide for scientific community for
precision tuning. and accuracy, see the surveys (Higham and Mary, 2022; Kashi et al. , 2025) and references therein. Overall, PROMISE automates the
derivation of mixed-precision mappings that would be difficult to obtain through manual analysis. 3 Customized precision with FloatX and its extension to mathematical functions
Most hardware implementations adhere to the IEEE 754 standard (IEEE Computer Society, 2019),
which has fixed allocations for sign, exponent, and significand bits. Overall, PROMISE automates the
derivation of mixed-precision mappings that would be difficult to obtain through manual analysis. 3 Customized precision with FloatX and its extension to mathematical functions
Most hardware implementations adhere to the IEEE 754 standard (IEEE Computer Society, 2019),
which has fixed allocations for sign, exponent, and significand bits. hilaire@lip6. fr
ORCID: 0000-0001-6365-9695
Fabienne Jézéquel*
Sorbonne Université, CNRS, LIP6
Paris, France
Université Paris-Panthéon-Assas
Paris, France
fabienne. Floating-point autotuning with customized precisions
Xinye Chen
Sorbonne Université, CNRS, LIP6
Paris, France
xinye. fr
ORCID: 0000-0003-1778-393X
Thibault Hilaire
Sorbonne Université, CNRS, LIP6
Paris, France
thibault. fr
ORCID: 0000-0002-8782-7566
Abstract
Reduced-precision arithmetic offers significant opportunities to improve performance, memory
usage, and energy efficiency in numerical applications, provided that numerical accuracy is preserved. This work investigates automated precision tuning through customized floating-point formats with
user-defined exponent and significand sizes, enabling the emulation of emerging low-precision
formats and the exploration of non-standard precision configurations within a unified mixed-precision
framework. , 2019) both based on the delta debugging algorithm (Zeller and Hildebrandt,
2002). CRAFT (Lam and Hollingsworth, 2018) uses binary instrumentation and modification to do
fine-grained precision analysis. , sin, cos, exp) operate in lowprecision environments by rounding inputs to the target precision before computation and rounding
outputs afterward, as detailed in Algorithm 1. Specifically, FloatX emulates custom floating-point formats
with user-defined exponent and significand bits, allowing fine-grained control over precision levels. Several approaches leverage automatic differentiation (AD) to estimate program output sensitivity to floating-point precision. ADAPT (Menon et al. previously mentioned for source-level mixed-precision tuning using AD-based analysis, static source transformations, and dynamic search methods. CHEF-FP (Singh et al. Existing work on precision tuning can be broadly categorized into
dynamic and static approaches (see Cherubin and Agosta (2020) and references therein) as well as
integrated mixed-precision toolchains (e. , FloatSmith (Lam et al. ADAPT (Menon et al. , 2018) uses algorithmic differentiation to guide
mixed-precision tuning and quantify accuracy impact, particularly in high-performance computing. Existing work on precision tuning can be broadly categorized into
dynamic and static approaches (see Cherubin and Agosta (2020) and references therein) as well as
integrated mixed-precision toolchains (e. , FloatSmith (Lam et al. Integrated mixed-precision
toolchains combine these static and dynamic techniques—often together with sensitivity analysis based on
automatic differentiation–to systematically explore, evaluate, and refine precision configurations across
the program. Dynamic analysis tools include the search-based tools Precimonious (Rubio-González et al. Integrated mixed-precision
toolchains combine these static and dynamic techniques—often together with sensitivity analysis based on
automatic differentiation–to systematically explore, evaluate, and refine precision configurations across
the program. Dynamic analysis tools include the search-based tools Precimonious (Rubio-González et al. previously mentioned for source-level mixed-precision tuning using AD-based analysis, static source transformations, and dynamic search methods. CHEF-FP (Singh et al. , 2023) integrates source-transformation
AD within LLVM/Clang for efficient, scalable error analysis. By embedding error-estimation logic into
automatically generated adjoint code, CHEF-FP enables rapid floating-point error analysis for largescale applications. FPTaylor (Solovyev et al. , 2019) establishes numerical error bounds
using symbolic Taylor expansions combined with global optimization, making it particularly effective for
verification-oriented floating-point programs. Only delta debugging (DD) and Genetic Search Algorithm (GA) can identify a valid
mixed precision configuration for all the benchmarks considered, DD producing slightly more performant
codes than GA. More recently, through a study of weather and climate models, the effectiveness of automated precision-tuning tools based on delta debugging was validated (Vanover et al. 2 PROMISE: dynamic precision analysis with numerical validation
2. 1 Discrete Stochastic Arithmetic
Using a probabilistic approach, the CESTAC method (Vignes, 1993) estimates the propagation of rounding
errors in floating-point computations by: (1) executing a given computational code N times, each with a
distinct rounding error propagation, and (2) deriving an estimate of the common component across these
executions, which serves as a reliable approximation of the exact result. Discrete Stochastic Arithmetic (DSA) (Vignes, 2004) combines the CESTAC method, the notion of
computational zero, and discrete stochastic relations. This framework enables the monitoring of scientific
computations, the detection of numerical instabilities, and the verification of the underlying assumptions,
thereby enhancing the robustness and reliability of numerical results. The approach is evaluated on a suite of numerical programs, including linear solvers
and applications from the Rodinia benchmark. Results show that a substantial proportion of variables
can be safely reduced to lower precision while preserving accuracy, indicating that standard double
precision is often over-provisioned. , 2013)
and PROMISE (Graillat et al. , 2019) both based on the delta debugging algorithm (Zeller and Hildebrandt,
2002). , 2013)
and PROMISE (Graillat et al. , 2019) both based on the delta debugging algorithm (Zeller and Hildebrandt,
2002).Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.
Improvements for AI systems
-
Automated mixed-precision derivation for linear solvers can be implemented by using PROMISE to
automate the derivation of mixed-precision mappings that would be difficult to obtain through manual analysis.
This allows AI systems solving linear systems to dynamically select the lowest precision formats, such as FP16 or BF16, that satisfy user-defined accuracy requirements. -
Enhanced neural network training stability can be achieved by using PROMISE to
identify locally optimal reduced-precision configurations in numerical software
and then applying these configurations to Back Propagation kernels. This results in models that require less memory and computational time while maintaining the required gradient accuracy, as demonstrated by findinga substantial proportion of variables can be safely reduced to lower precision while preserving accuracy.
-
Improved performance for physics simulations, such as those in HotSpot thermal simulation, can be realized by using PROMISE to explore configurations where
the dominant error has shifted from local rounding to accumulated bias in the global time-step scaling.
This enables the system to correctly identify when promoting variables like cell capacitance C or update terms to FP32 or BF16 yields a meaningful reduction in error. -
Robust particle tracking systems can benefit from using PROMISE's
delta debugging algorithm
across multiple precision types to ensure accuracy requirements are met during Monte Carlo localization. This systematic search identifies thelocally minimal set of program locations that must retain higher precision, enabling precision reduction elsewhere,
leading to faster and more reliable trajectory computations. -
Custom low-precision mathematical functions can be integrated into AI kernels by using FloatX extensions to emulate user-defined formats for functions like sin or exp. This allows AI systems to
emulate custom floating-point formats with user-defined exponent and significand bits, allowing fine-grained control over precision levels
during intermediate calculations.
Abstract
Reduced-precision arithmetic offers significant opportunities to improve performance, memory usage, and energy efficiency in numerical applications, provided that numerical accuracy is preserved. This work investigates automated precision tuning through customized floating-point formats with user-defined exponent and significand sizes, enabling the emulation of emerging low-precision formats and the exploration of non-standard precision configurations within a unified mixed-precision framework. The proposed methodology, implemented in the PROMISE precision autotuning tool, combines numerical validation with a systematic search to generate program variants that satisfy user-defined accuracy requirements. To address the computational cost of this exploration, a containerized benchmarking framework supports parallel execution across multiple algorithms and parameter configurations. The approach is evaluated on a suite of numerical programs, including linear solvers and applications from the Rodinia benchmark. Results show that a substantial proportion of variables can be safely reduced to lower precision while preserving accuracy, indicating that standard double precision is often over-provisioned. These findings highlight the potential of automated precision tuning to derive efficient mixed-precision configurations tailored to application-specific accuracy requirements.
Sources
Related papers
- asdex: Automatic Sparse Differentiation in JAX
- Lord of the Precision, the Two Floats: Double-Word Arithmetic for Complex Numbers
- Streaming Right Multiplication over Grammar-Compressed Matrices: A Memory-Bounded GPU Engine for Genotype and Graph Data
- Improved Scaling for Fast Mode of Ozaki Scheme II
- BF16 Component-Product Emulation of FP32 and FP64 GEMM on Intel AMX
- QRecoupling.jl: numerical and exact computation of classical and quantum recoupling coefficients