Floating-point autotuning with customized precisions

arXiv:2606.08339 · cs.MS, cs.NA, math.NA · Submitted 2026-06-06 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: Computer Science. The day in computer science: software, systems and theory on arXiv.

Tom: We're looking at the paper: "Floating-point autotuning with customized precisions".

Jane: Here's the gist of the paper. Reduced-precision arithmetic offers significant opportunities to improve performance, memory usage, and energy efficiency in numerical applications, provided that numerical accuracy is preserved<ref:2606.08339#pg2> This work investigates automated precision tuning through customized floating-point formats with user-defined exponent and significand sizes, enabling the emulation of emerging low-precision formats and the exploration of non-standard precision configurations within a unified mixed-precision framework<ref:2606.08339#pg3> The proposed methodology, implemented in the PROMISE precision autotuning tool, combines numerical validation with a systematic search to generate program variants that satisfy userdefined accuracy requirements<ref:2606.08339#pg5> Results show that a substantial proportion of variables can be safely reduced to lower precision while preserving accuracy, indicating that standard double precision is often over-provisioned<ref:2606.08339#pg6> These findings highlight the potential of automated precision tuning to derive efficient mixed-precision configurations tailored to application-specific accuracy requirements<ref:2606.08339#pg6> The approach involves a systematic search guided by numerical validation<ref:2606.08339#pg5> PROMISE operates on C and C++ programs, requiring the user to provide the code under investigation along with annotations that specify precision-tuning candidates and the desired accuracy of the results<ref:2606.08339#pg8> The tool first transforms the input program into an instrumented version using CADNAIZER, which is a source-to-source transformation<ref:2606.08339#pg9> This instrumented code is then executed with the CADNA library, which employs Discrete Stochastic Arithmetic to estimate the number of exact significant digits and produce numerically validated reference results<ref:2606.08339#pg10> Based on this information, PROMISE explores the space of mixed-precision configurations using a delta debugging algorithm The search starts from a high-precision baseline and progressively lowers precision while discarding configurations that violate numerical correctness When a given configuration is validated, the search continues along this direction without exploring alternative configurations at the same level, thereby avoiding an exhaustive exploration of the search space Customized Precision Formats The paper introduces customization by extending PROMISE to support customized low-precision formats FloatX is a C++ header-only library designed to emulate reduced-precision floating-point formats with user-specific bit-lengths for the exponent and the significand The format definition uses the notation flx::floatx⟨e, t⟩ ≡ 1 + e + t bits Sign where e is the biased exponent and t is the trailing significand PROMISE overloads standard mathematical functions and ensures that all intermediate computations adhere to the specified reduced-precision formats This approach avoids the limitations of tools that rely on predefined internal precisions, providing a flexible and efficient solution for low-precision mathematical function emulation Benchmarking and Validation The deployment relies on a containerized benchmarking tool that supports parallel execution across multiple algorithms and parameter configurations The framework scales with available computing resources, permitting efficient utilization of multicore CPUs and cluster nodes for large benchmark suites The total runtime for precision tuning across the benchmarks drops from about 13 hours to 3 hours, resulting in a roughly 3.4× speedup The study evaluates algorithms from the Rodinia benchmark suite and well-established linear solving methods Empirical Findings on Precision Reduction Averaged over all benchmarks and target accuracy levels, 43% of the variables can be represented with 32-bit precision or less, and 28% of the variables can be reduced to 16-bit precision or lower These results indicate that FP64 is systematically over-provisioned for the levels of accuracy considered Customized precision enables one to find suitable type configurations that include low precision formats available on the computing target The use of FP32 tends to align with a small number of sensitivity bottlenecks (e.g., exponentials/logs, variance-like reductions, and weight/CDF accumulation paths) Algorithm Specific Sensitivity Analysis The analysis reveals distinct sensitivity patterns across different numerical algorithms For Backprop, precision upgrades concentrate first on the sigmoid pre-activation sums and the activation path, then on the error-delta computations, and finally on the weight-update accumulators when a higher number of digits is demanded In HotSpot, more variables, including important thermal coefficients such as C and ∆, are promoted to FP32 for 7 to 10 required digits The Particle Filter exhibits a mixed-precision sensitivity pattern driven primarily by control-flow decisions rather than long reduction chains For SRAD, most of the precision challenges appear in the coefficient pathway, including Jc normalization, forming the normalized directional measures, and the later rational and nonlinear steps that lead to q2 and c In this paper, we advance PROMISE by incorporating customized-precision emulation and enabling scalable deployment Our empirical results show that PROMISE enables substantial precision reductions while meeting the required accuracy, which can in turn lead to performance improvements The extension of PROMISE to GPUs is a planned perspective that would benefit from the GPU version of CADNA Future work will also focus on precision autotuning for iterative linear system solvers, with particular attention to potential increases in iteration count when low precision is applied to automatically selected variables The contribution to the PROMISE software is integrated into https://gitlab.lip6.fr/pequan/promise References Dorra Ben Khalifa, Matthieu Martel, and Assalé Adjé.

Jane: POP: A Tuning Assistant for MixedPrecision Floating-Point Computations<ref:2606.08339#pg2> Shuai Che, Michael Boyer, Jiayuan Meng, David Tarjan, Jeremy W.

Tom: Sheaffer, Sang-Ha Lee, and Kevin Skadron.

Title and authors: Tom: Now moving into the specific details of the title, "Floating-point autotuning with customized precisions." This paper is really about making a decision about precision—which floating point type to use—a problem that software can solve automatically rather than someone having to guess.

Jane: It’s essentially saying that we can design custom formats, defining user-specified exponent and significand sizes, which lets the tool emulate those emerging low-precision formats we see coming in the industry today.

Lu: The core idea is using PROMISE to explore those non-standard precision configurations within a unified mixed precision framework, which means it handles different precision types all at once.

Meng: But that sounds complex. How does this customized format thing actually translate into something that runs efficiently on a typical machine instead of just being some theoretical exercise?

Lalam: It translates into PROMISE overloading standard mathematical functions to ensure all intermediate computations adhere to these specified reduced precision formats, which avoids the limitations of tools relying on predefined internal precisions.

Tom: That means you aren't stuck with what the compiler or library decides is "good enough"; you can dictate the exact bit-length for both the exponent and the significand.

Jane: And this whole framework relies on this methodology—the systematic search guided by numerical validation—to find those optimal configurations that match user-defined accuracy requirements.

Tom: So, if I have a specific accuracy target, PROMISE searches through the space of possible mixed precision settings until it finds one that works.

Lu: The paper highlights that this approach is a way to address the computational cost of error estimation by integrating delta debugging into dynamic search methods like genetic search algorithms or even six other search algorithms they compared in previous work.

Meng: What about the practical cost? If you’re running a massive simulation, how much extra overhead does this autotuning process itself add before you even get to the actual computation?

Lalam: The authors benchmarked this, and their tuning process across benchmarks dropped from about thirteen hours down to three hours, which is a speedup of roughly three point four times.

The paper's summary: Jane: So we touched on the setup, but let's get into what they actually did. The paper describes a methodology where PROMISE first transforms the input program into an instrumented version using CADNAIZER, which is a source-to-source transformation.

Tom: That CADNAIZER step is crucial because it sets up the code so that we can then execute it with the CADNA library to estimate the number of exact significant digits and produce numerically validated reference results.

Lu: This numerical validation gives them the data they need to guide their search process, which they implement using a delta debugging algorithm.

Meng: Delta debugging sounds intense. So, how does that actually work in practice when you're trying to find optimal precision settings? What is the trade-off there?

Lalam: The delta debugging algorithm starts from a high precision baseline and progressively lowers the precision while discarding configurations that violate numerical correctness, continuing along this direction without exploring alternative configurations at the same level.

Tom: That’s a very smart way to search because it avoids an exhaustive exploration of the entire search space, which is something they acknowledge is impractical for large code bases.

Jane: The paper shows that this method successfully identifies a valid mixed precision configuration for all the benchmarks they considered, and delta debugging produced slightly more performant codes than the genetic search algorithm.

Lu: They also validated this effectiveness through a study of weather and climate models, showing that these automated precision-tuning tools based on delta debugging are effective.

Meng: So, if I'm running production workloads, what kind of program would I even feed into PROMISE to test this out? Are we talking about deep learning training or something more scientific?

Lalam: They evaluated algorithms from the Rodinia benchmark suite and well-established linear solving methods.

The paper's improvements: Tom: Moving on to how they improve things, the biggest change here is extending PROMISE to support customized low-precision formats using FloatX. This gives them a way to define formats with user-specific bit-lengths for both the exponent and the significand.

Jane: That customization is what allows PROMISE to avoid the limitations of tools that rely on predefined internal precisions, offering a much more flexible solution for emulating those emerging low precision formats.

Lu: The paper also shows how they integrate this with source-level mixed precision tuning using AD-based analysis and static source transformations, similar to what CHEF-FP does, but their application focuses on a search-based tool based on delta debugging.

Meng: So they’re showing that you can get better results by using AI techniques for error analysis and then feeding those findings into a systematic search process. That sounds like it could be very powerful for optimizing complex neural network training paths.

Lalam: The empirical findings show that averaged over all benchmarks and target accuracy levels, forty-three percent of the variables can be represented with thirty-two-bit precision or less, and twenty-eight percent of the variables can be reduced to sixteen-bit precision or lower.

Tom: That's a concrete number. It means standard FP64 is systematically over-provisioned for the levels of accuracy they were testing at, which is a big statement about where we are in numerical computing.

Jane: And they also found that the use of FP32 tends to align with a small number of sensitivity bottlenecks, like exponentials and logs, variance-like reductions, and weight accumulation paths.

Conclusion: Tom: So to wrap up this paper on "Floating-point autotuning with customized precisions," the main implication is that PROMISE enables substantial precision reductions while meeting the required accuracy, which directly leads to performance improvements in numerical applications.

Jane: It means we can move towards more efficient mixed precision configurations tailored specifically to the accuracy needs of a particular application rather than using one fixed setting for everything.

Lu: The paper advances PROMISE by incorporating customized-precision emulation and enabling scalable deployment, with a planned perspective to extend PROMISE to GPUs using the GPU version of CADNA.

Meng: From an engineering standpoint, the ability to find locally optimal reduced precision configurations in numerical software can lead to models that require less memory and computational time while maintaining the required gradient accuracy.

Lalam: And they also focus future work on precision autotuning for iterative linear system solvers, paying attention to potential increases in iteration count when low precision is applied to automatically selected variables.

Tom: It sounds like this paper lays a lot of groundwork for finding those specific bottlenecks that matter most for any given numerical problem.

Jane: It’s a solid piece of work showing how automated tools can derive efficient mixed precision configurations tailored to application-specific accuracy requirements, which is exactly what we need when we're pushing the limits on HPC.

Xinye Chen, Thibault Hilaire, §Fabienne JĞquʂel

Sorbonne University · CNRS

cs.MS, cs.NA, math.NA

Submitted: 2026-06-06

Updated: 2026-10-29

Code: https://github.com/oprecomp/FloatX

License: http://creativecommons.org/licenses/by/4.0/

Importance score: 76/100

The gist: Reduced-precision arithmetic offers significant opportunities to improve performance, memory usage, and energy efficiency in numerical applications…

Key concepts

PROMISE
PROMISE is an automated tool designed to tune the precision of numerical programs. It works by transforming the code, validating results using a library that estimates exact digits, and then systematically searching for optimal mixed-precision configurations that meet user accuracy goals.
Customized Precision Formats (FloatX)
This feature allows users to define their own reduced-precision floating-point formats by specifying custom sizes for the exponent and significand bits. PROMISE uses this to emulate emerging low-precision standards, giving fine-grained control over the precision used in intermediate calculations.
Delta Debugging Algorithm
This is the search method PROMISE employs to find optimal configurations. It starts with a high-precision baseline and progressively lowers precision. Crucially, it only explores paths that maintain numerical correctness, avoiding an exhaustive search by moving systematically along validated directions.
Mixed-Precision Framework
This framework allows a program to use different floating-point precisions for different parts of the computation. PROMISE manages this by ensuring that all intermediate mathematical functions adhere strictly to the specified reduced-precision formats, enabling efficient resource utilization.

Terminology

Summary

Reduced-precision arithmetic offers significant opportunities to improve performance, memory usage, and energy efficiency in numerical applications, provided that numerical accuracy is preserved<ref:2606. p. 0

Where the paper says it

jezequel@lip6. fr ORCID: 0000-0002-8782-7566 Abstract Reduced-precision arithmetic offers significant opportunities to improve performance, memory usage, and energy efficiency in numerical applications, provided that numerical accuracy is preserved.

08339#pg2> This work investigates automated precision tuning through customized floating-point formats with user-defined exponent and significand sizes, enabling the emulation of emerging low-precision formats and the exploration of non-standard precision configurations within a unified mixed-precision framework<ref:2606. p. 0
Where the paper says it

fr ORCID: 0000-0002-8782-7566 Abstract Reduced-precision arithmetic offers significant opportunities to improve performance, memory usage, and energy efficiency in numerical applications, provided that numerical accuracy is preserved. This work investigates automated precision tuning through customized floating-point formats with user-defined exponent and significand sizes, enabling the emulation of emerging low-precision formats and the exploration of non-standard precision configurations within a unified mixed-precision framework.

08339#pg3> The proposed methodology, implemented in the PROMISE precision autotuning tool, combines numerical validation with a systematic search to generate program variants that satisfy userdefined accuracy requirements<ref:2606. p. 0
Where the paper says it

This work investigates automated precision tuning through customized floating-point formats with user-defined exponent and significand sizes, enabling the emulation of emerging low-precision formats and the exploration of non-standard precision configurations within a unified mixed-precision framework. The proposed methodology, implemented in the PROMISE precision autotuning tool, combines numerical validation with a systematic search to generate program variants that satisfy userdefined accuracy requirements.

08339#pg5> Results show that a substantial proportion of variables can be safely reduced to lower precision while preserving accuracy, indicating that standard double precision is often over-provisioned<ref:2606. p. 0
Where the paper says it

The approach is evaluated on a suite of numerical programs, including linear solvers and applications from the Rodinia benchmark. Results show that a substantial proportion of variables can be safely reduced to lower precision while preserving accuracy, indicating that standard double precision is often over-provisioned.

08339#pg6> These findings highlight the potential of automated precision tuning to derive efficient mixed-precision configurations tailored to application-specific accuracy requirements<ref:2606. p. 0
Where the paper says it

Results show that a substantial proportion of variables can be safely reduced to lower precision while preserving accuracy, indicating that standard double precision is often over-provisioned. These findings highlight the potential of automated precision tuning to derive efficient mixed-precision configurations tailored to application-specific accuracy requirements.

08339#pg6>

Methodology for Precision Tuning

The approach involves a systematic search guided by numerical validation<ref:2606.08339#pg5> PROMISE operates on C and C++ programs, requiring the user to provide the code under investigation along with annotations that specify precision-tuning candidates and the desired accuracy of the results<ref:2606. p. 3

Where the paper says it

2 Automated precision tuning with Discrete Stochastic Arithmetic PROMISE is a software designed to identify locally optimal reduced-precision configurations in numerical software by integrating DSA, code instrumentation, and automated source-to-source transformations. It operates on C and C++ programs, requiring the user to provide the code under investigation along with annotations that specify precision-tuning candidates and the desired accuracy of the results.

08339#pg8> The tool first transforms the input program into an instrumented version using CADNAIZER, which is a source-to-source transformation<ref:2606. p. 3
Where the paper says it

It operates on C and C++ programs, requiring the user to provide the code under investigation along with annotations that specify precision-tuning candidates and the desired accuracy of the results. PROMISE first transforms the input program into an instrumented version using the CADNAIZER tool.

08339#pg9> This instrumented code is then executed with the CADNA library, which employs Discrete Stochastic Arithmetic to estimate the number of exact significant digits and produce numerically validated reference results<ref:2606. p. 3
Where the paper says it

PROMISE first transforms the input program into an instrumented version using the CADNAIZER tool. This instrumented code is then executed with the CADNA library, which employs Discrete Stochastic Arithmetic to estimate the number of exact significant digits and produce numerically validated reference results.

08339#pg10> Based on this information, PROMISE explores the space of mixed-precision configurations using a delta debugging algorithm The search starts from a high-precision baseline and progressively lowers precision while discarding configurations that violate numerical correctness When a given configuration is validated, the search continues along this direction without exploring alternative configurations at the same level, thereby avoiding an exhaustive exploration of the search space

Customized Precision Formats

The paper introduces customization by extending PROMISE to support customized low-precision formats FloatX is a C++ header-only library designed to emulate reduced-precision floating-point formats with user-specific bit-lengths for the exponent and the significand The format definition uses the notation flx::floatx⟨e, t⟩ ≡ 1 + e + t bits Sign where e is the biased exponent and t is the trailing significand PROMISE overloads standard mathematical functions and ensures that all intermediate computations adhere to the specified reduced-precision formats This approach avoids the limitations of tools that rely on predefined internal precisions, providing a flexible and efficient solution for low-precision mathematical function emulation

Benchmarking and Validation

The deployment relies on a containerized benchmarking tool that supports parallel execution across multiple algorithms and parameter configurations The framework scales with available computing resources, permitting efficient utilization of multicore CPUs and cluster nodes for large benchmark suites The total runtime for precision tuning across the benchmarks drops from about 13 hours to 3 hours, resulting in a roughly 3.4× speedup The study evaluates algorithms from the Rodinia benchmark suite and well-established linear solving methods

Empirical Findings on Precision Reduction

Averaged over all benchmarks and target accuracy levels, 43% of the variables can be represented with 32-bit precision or less, and 28% of the variables can be reduced to 16-bit precision or lower These results indicate that FP64 is systematically over-provisioned for the levels of accuracy considered Customized precision enables one to find suitable type configurations that include low precision formats available on the computing target The use of FP32 tends to align with a small number of sensitivity bottlenecks (e.g., exponentials/logs, variance-like reductions, and weight/CDF accumulation paths)

Algorithm Specific Sensitivity Analysis

The analysis reveals distinct sensitivity patterns across different numerical algorithms For Backprop, precision upgrades concentrate first on the sigmoid pre-activation sums and the activation path, then on the error-delta computations, and finally on the weight-update accumulators when a higher number of digits is demanded In HotSpot, more variables, including important thermal coefficients such as C and ∆, are promoted to FP32 for 7 to 10 required digits The Particle Filter exhibits a mixed-precision sensitivity pattern driven primarily by control-flow decisions rather than long reduction chains For SRAD, most of the precision challenges appear in the coefficient pathway, including Jc normalization, forming the normalized directional measures, and the later rational and nonlinear steps that lead to q2 and c

Conclusion and Future Directions

In this paper, we advance PROMISE by incorporating customized-precision emulation and enabling scalable deployment Our empirical results show that PROMISE enables substantial precision reductions while meeting the required accuracy, which can in turn lead to performance improvements The extension of PROMISE to GPUs is a planned perspective that would benefit from the GPU version of CADNA Future work will also focus on precision autotuning for iterative linear system solvers, with particular attention to potential increases in iteration count when low precision is applied to automatically selected variables The contribution to the PROMISE software is integrated into https://gitlab.lip6.fr/pequan/promise

References

Dorra Ben Khalifa, Matthieu Martel, and Assalé Adjé. 2020. POP: A Tuning Assistant for MixedPrecision Floating-Point Computations<ref:2606. p. 0

Where the paper says it

Attaining optimal performance in numerical algorithms can often be achieved by quantizing floating-point numbers to low-precision formats while conducting meticulous rounding error analysis to maintain numerical stability *This work was supported by the France 2030 NumPEx Exa-MA project (ANR-22-EXNU-0002) managed by the French National Research Agency (ANR). 1 arXiv:2606.

08339#pg2> Shuai Che, Michael Boyer, Jiayuan Meng, David Tarjan, Jeremy W. p. 8
Where the paper says it

Our simulation tasks as well as their categorization are presented in Table 2. PROMISE is run on algorithms from the Rodinia benchmark suite6 (Che et al.

Sheaffer, Sang-Ha Lee, and Kevin Skadron. 2009. Rodinia: A benchmark suite for heterogeneous computing<ref:2606. p. 0
Where the paper says it

To address the computational cost of this exploration, a containerized benchmarking framework supports parallel execution across multiple algorithms and parameter configurations. The approach is evaluated on a suite of numerical programs, including linear solvers and applications from the Rodinia benchmark.

08339#pg2> Stefano Cherubin and Giovanni Agosta. p. 1
Where the paper says it

However, this method is not practical for large code base. Existing work on precision tuning can be broadly categorized into dynamic and static approaches (see Cherubin and Agosta (2020) and references therein) as well as integrated mixed-precision toolchains (e.

2020. Tools for Reduced Precision Computation: A Survey<ref:2606. p. 0
Where the paper says it

1 Introduction Performing arithmetic with lower precision usually leads to less computational cost, energy consumption and communication cost. With the prevalence of hardware-supported reduced-precision arithmetic, many algorithms can be sped up by hardware accelerators using low-precision arithmetic, as high precision is often unnecessary for every step of their computation.

08339#pg8> Wei-Fan Chiang, Mark Baranowski, Ian Briggs, Alexey Solovyev, Ganesh Gopalakrishnan, and Zvonimir Rakamaric. 2017. Rigorous floating-point mixed-precision tuning Timothy A. p. 0
Where the paper says it

fr ORCID: 0000-0002-8782-7566 Abstract Reduced-precision arithmetic offers significant opportunities to improve performance, memory usage, and energy efficiency in numerical applications, provided that numerical accuracy is preserved. This work investigates automated precision tuning through customized floating-point formats with user-defined exponent and significand sizes, enabling the emulation of emerging low-precision formats and the exploration of non-standard precision configurations within a unified mixed-precision framework.

Davis and Yifan Hu. p. 9
Where the paper says it

For the dense solver, we use a synthetically generated matrix with a condition number of 104 . For the sparse solver, we use the sherman1 matrix from the SuiteSparse Matrix Collection Davis and Hu (2011), which has a condition number of 1.

2011. The University of Florida sparse matrix collection<ref:2606. p. 9
Where the paper says it

For the dense solver, we use a synthetically generated matrix with a condition number of 104 . For the sparse solver, we use the sherman1 matrix from the SuiteSparse Matrix Collection Davis and Hu (2011), which has a condition number of 1.

08339#pg8> P. Eberhart, J. p. 2
Where the paper says it

Previously mentioned, the precision autotuning tool PROMISE (PRecision OptiMISE)1 (Graillat et al. , 2019) leverages the CADNA library (Jézéquel and Chesneaux, 2008; Eberhart et al.

Brajard, P. Fortin, and F. Jézéquel. p. 0
Where the paper says it

hilaire@lip6. fr ORCID: 0000-0001-6365-9695 Fabienne Jézéquel* Sorbonne Université, CNRS, LIP6 Paris, France Université Paris-Panthéon-Assas Paris, France fabienne.

2015. High Performance Numerical Validation using Stochastic Arithmetic<ref:2606. p. 3
Where the paper says it

2 PROMISE: dynamic precision analysis with numerical validation 2. 1 Discrete Stochastic Arithmetic Using a probabilistic approach, the CESTAC method (Vignes, 1993) estimates the propagation of rounding errors in floating-point computations by: (1) executing a given computational code N times, each with a distinct rounding error propagation, and (2) deriving an estimate of the common component across these executions, which serves as a reliable approximation of the exact result.

08339#pg8> Quentin Ferro, Stef Graillat, Thibault Hilaire, and Fabienne Jézéquel. p. 0
Where the paper says it

hilaire@lip6. fr ORCID: 0000-0001-6365-9695 Fabienne Jézéquel* Sorbonne Université, CNRS, LIP6 Paris, France Université Paris-Panthéon-Assas Paris, France fabienne.

2023. Performance of precision autotuned neural networks Goran Flegar, Florian Scheidegger, Vedran Novakovic, Giovani Mariani, Andrés E. p. 2
Where the paper says it

(2021) extended this support to FP16 precision, while subsequent work Ferro et al. (2023) applied PROMISE to the precision tuning of neural networks.

Tomás, A. Cristiano I. Malossi, and Enrique S. Quintana-Ortí. 2019. FloatX: A C++ Library for Customized Floating-Point Arithmetic Laurent Fousse, Guillaume Hanrot, Vincent Lefèvre, Patrick Pélissier, and Paul Zimmermann. p. 4
Where the paper says it

FloatX (Float eXtended3 ) (Flegar et al. , 2019) is a C++ header-only library designed to emulate reduced-precision floating-point formats with user-specific bit-lengths for the exponent and the significand, and to facilitate incremental code transformation.

2007. MPFR: A multiple-precision binary floating-point library with correct rounding Gene H. p. 3
Where the paper says it

CADNA2 (Jézéquel and Chesneaux, 2008) is a library that applies DSA to track rounding error effects. It requires users to replace ordinary floating-point variables with its own stochastic types, which means a few declaration changes in C or C++ code.

Golub and Charles F. p. 9
Where the paper says it

Structured grid, compute-intensive, iterative. LU Factorization Golub and Van Loan (2013) Linear Algebra Matrix decomposition via Gaussian elimination for solving linear systems.

Van Loan. 2013. Matrix Computations (4 ed. p. 3
Where the paper says it

2 PROMISE: dynamic precision analysis with numerical validation 2. 1 Discrete Stochastic Arithmetic Using a probabilistic approach, the CESTAC method (Vignes, 1993) estimates the propagation of rounding errors in floating-point computations by: (1) executing a given computational code N times, each with a distinct rounding error propagation, and (2) deriving an estimate of the common component across these executions, which serves as a reliable approximation of the exact result.

). Johns Hopkins University Press, BaltimoreStef Graillat, Fabienne Jézéquel, Romain Picot, François Févotte, and Bruno Lathuilière. p. 0
Where the paper says it

hilaire@lip6. fr ORCID: 0000-0001-6365-9695 Fabienne Jézéquel* Sorbonne Université, CNRS, LIP6 Paris, France Université Paris-Panthéon-Assas Paris, France fabienne.

2019. Autotuning for floating-point precision with Discrete Stochastic Arithmetic Hui Guo and Cindy Rubio-González. p. 3
Where the paper says it

2 PROMISE: dynamic precision analysis with numerical validation 2. 1 Discrete Stochastic Arithmetic Using a probabilistic approach, the CESTAC method (Vignes, 1993) estimates the propagation of rounding errors in floating-point computations by: (1) executing a given computational code N times, each with a distinct rounding error propagation, and (2) deriving an estimate of the common component across these executions, which serves as a reliable approximation of the exact result.

2018. Exploiting community structure for floating-point precision tuning Nicholas J. p. 2
Where the paper says it

We validate our apporach with extensive simulations on algorithms for numerical approximation and linear system solving. We believe that this study can offer a guide for scientific community for precision tuning.

Higham and Theo Mary. p. 1
Where the paper says it

and accuracy, see the surveys (Higham and Mary, 2022; Kashi et al. , 2025) and references therein.

2022. Mixed precision algorithms in numerical linear algebraIEEE Computer Society. p. 4
Where the paper says it

Overall, PROMISE automates the derivation of mixed-precision mappings that would be difficult to obtain through manual analysis. 3 Customized precision with FloatX and its extension to mathematical functions Most hardware implementations adhere to the IEEE 754 standard (IEEE Computer Society, 2019), which has fixed allocations for sign, exponent, and significand bits.

2019. IEEE Standard for Floating-Point ArithmeticF. p. 4
Where the paper says it

Overall, PROMISE automates the derivation of mixed-precision mappings that would be difficult to obtain through manual analysis. 3 Customized precision with FloatX and its extension to mathematical functions Most hardware implementations adhere to the IEEE 754 standard (IEEE Computer Society, 2019), which has fixed allocations for sign, exponent, and significand bits.

Jézéquel and J. p. 0
Where the paper says it

hilaire@lip6. fr ORCID: 0000-0001-6365-9695 Fabienne Jézéquel* Sorbonne Université, CNRS, LIP6 Paris, France Université Paris-Panthéon-Assas Paris, France fabienne.

-M. Chesneaux. 2008. CADNA: A Library for Estimating Round-Off Error PropagationFabienne Jézéquel, Sara sadat Hoseininasab, and Thibault Hilaire. p. 0
Where the paper says it

Floating-point autotuning with customized precisions Xinye Chen Sorbonne Université, CNRS, LIP6 Paris, France xinye. fr ORCID: 0000-0003-1778-393X Thibault Hilaire Sorbonne Université, CNRS, LIP6 Paris, France thibault.

2021. Numerical Validation of Half Precision SimulationsAditya Kashi, Hao Lu, Wesley Brewer, David Rogers, Michael Matheson, Mallikarjun Shankar, and Feiyi Wang. 2025. Mixed-precision numerics in scientific applications: survey and perspectives<ref:2606. p. 0
Where the paper says it

fr ORCID: 0000-0002-8782-7566 Abstract Reduced-precision arithmetic offers significant opportunities to improve performance, memory usage, and energy efficiency in numerical applications, provided that numerical accuracy is preserved. This work investigates automated precision tuning through customized floating-point formats with user-defined exponent and significand sizes, enabling the emulation of emerging low-precision formats and the exploration of non-standard precision configurations within a unified mixed-precision framework.

08339#pg8>Michael O Lam and Jeffrey K Hollingsworth. p. 1
Where the paper says it

, 2019) both based on the delta debugging algorithm (Zeller and Hildebrandt, 2002). CRAFT (Lam and Hollingsworth, 2018) uses binary instrumentation and modification to do fine-grained precision analysis.

2018. Fine-grained floating-point precision analysisMichael O. p. 5
Where the paper says it

, sin, cos, exp) operate in lowprecision environments by rounding inputs to the target precision before computation and rounding outputs afterward, as detailed in Algorithm 1. Specifically, FloatX emulates custom floating-point formats with user-defined exponent and significand bits, allowing fine-grained control over precision levels.

Lam, Tristan Vanderbruggen, Harshitha Menon, and Markus Schordan. p. 1
Where the paper says it

Several approaches leverage automatic differentiation (AD) to estimate program output sensitivity to floating-point precision. ADAPT (Menon et al.

2019. Tool Integration for Source-Level Mixed PrecisionHarshitha Menon, Michael O. p. 2
Where the paper says it

previously mentioned for source-level mixed-precision tuning using AD-based analysis, static source transformations, and dynamic search methods. CHEF-FP (Singh et al.

Lam, Daniel Osei-Kuffuor, Markus Schordan, Scott Lloyd, Kathryn Mohror, and Jeffrey Hittinger. p. 1
Where the paper says it

Existing work on precision tuning can be broadly categorized into dynamic and static approaches (see Cherubin and Agosta (2020) and references therein) as well as integrated mixed-precision toolchains (e. , FloatSmith (Lam et al.

2018. ADAPT: Algorithmic Differentiation Applied to Floating-Point Precision TuningKonstantinos Parasyris, Ignacio Laguna, Harshitha Menon, Markus Schordan, Daniel Osei-Kuffuor, Giorgis Georgakoudis, Michael O. p. 1
Where the paper says it

ADAPT (Menon et al. , 2018) uses algorithmic differentiation to guide mixed-precision tuning and quantify accuracy impact, particularly in high-performance computing.

Lam, and Tristan Vanderbruggen. p. 1
Where the paper says it

Existing work on precision tuning can be broadly categorized into dynamic and static approaches (see Cherubin and Agosta (2020) and references therein) as well as integrated mixed-precision toolchains (e. , FloatSmith (Lam et al.

2020. HPC-MixPBench: An HPC Benchmark Suite for Mixed-Precision AnalysisCindy Rubio-González, Cuong Nguyen, Hong Diep Nguyen, James Demmel, William Kahan, Koushik Sen, David H. p. 1
Where the paper says it

Integrated mixed-precision toolchains combine these static and dynamic techniques—often together with sensitivity analysis based on automatic differentiation–to systematically explore, evaluate, and refine precision configurations across the program. Dynamic analysis tools include the search-based tools Precimonious (Rubio-González et al.

Bailey, Costin Iancu, and David Hough. 2013. Precimonious: tuning assistant for floatingpoint precisionCindy Rubio-González, Cuong Nguyen, Benjamin Mehne, Koushik Sen, James Demmel, William Kahan, Costin Iancu, Wim Lavrijsen, David H. p. 1
Where the paper says it

Integrated mixed-precision toolchains combine these static and dynamic techniques—often together with sensitivity analysis based on automatic differentiation–to systematically explore, evaluate, and refine precision configurations across the program. Dynamic analysis tools include the search-based tools Precimonious (Rubio-González et al.

Bailey, and David Hough. 2016. Floating-Point Precision Tuning Using Blame AnalysisGarima Singh, Baidyanath Kundu, Harshitha Menon, Alexander Penev, David J. p. 2
Where the paper says it

previously mentioned for source-level mixed-precision tuning using AD-based analysis, static source transformations, and dynamic search methods. CHEF-FP (Singh et al.

Lange, and Vassil Vassilev. 2023. Fast and Automatic Floating-Point Error Analysis with CHEF-FPAlexey Solovyev, Marek S. p. 2
Where the paper says it

, 2023) integrates source-transformation AD within LLVM/Clang for efficient, scalable error analysis. By embedding error-estimation logic into automatically generated adjoint code, CHEF-FP enables rapid floating-point error analysis for largescale applications.

Baranowski, Ian Briggs, Charles Jacobsen, Zvonimir Rakamaric, and Ganesh Gopalakrishnan. 2019. Rigorous Estimation of Floating-Point Round-Off Errors with Symbolic Taylor ExpansionsJackson Vanover, Alper Altuntas, and Cindy Rubio-González. p. 1
Where the paper says it

FPTaylor (Solovyev et al. , 2019) establishes numerical error bounds using symbolic Taylor expansions combined with global optimization, making it particularly effective for verification-oriented floating-point programs.

2024. Toward Automated Precision Tuning of Weather and Climate Models: A Case StudyJean Vignes. p. 2
Where the paper says it

Only delta debugging (DD) and Genetic Search Algorithm (GA) can identify a valid mixed precision configuration for all the benchmarks considered, DD producing slightly more performant codes than GA. More recently, through a study of weather and climate models, the effectiveness of automated precision-tuning tools based on delta debugging was validated (Vanover et al.

1993. A stochastic arithmetic for reliable scientific computationJean Vignes. p. 3
Where the paper says it

2 PROMISE: dynamic precision analysis with numerical validation 2. 1 Discrete Stochastic Arithmetic Using a probabilistic approach, the CESTAC method (Vignes, 1993) estimates the propagation of rounding errors in floating-point computations by: (1) executing a given computational code N times, each with a distinct rounding error propagation, and (2) deriving an estimate of the common component across these executions, which serves as a reliable approximation of the exact result.

2004. Discrete Stochastic Arithmetic for Validating Results of Numerical SoftwareYutong Wang and Cindy Rubio-González. p. 3
Where the paper says it

Discrete Stochastic Arithmetic (DSA) (Vignes, 2004) combines the CESTAC method, the notion of computational zero, and discrete stochastic relations. This framework enables the monitoring of scientific computations, the detection of numerical instabilities, and the verification of the underlying assumptions, thereby enhancing the robustness and reliability of numerical results.

2024. Predicting Performance and Accuracy of MixedPrecision Programs for Precision TuningA. p. 0
Where the paper says it

The approach is evaluated on a suite of numerical programs, including linear solvers and applications from the Rodinia benchmark. Results show that a substantial proportion of variables can be safely reduced to lower precision while preserving accuracy, indicating that standard double precision is often over-provisioned.

Zeller and R. p. 1
Where the paper says it

, 2013) and PROMISE (Graillat et al. , 2019) both based on the delta debugging algorithm (Zeller and Hildebrandt, 2002).

Hildebrandt. p. 1
Where the paper says it

, 2013) and PROMISE (Graillat et al. , 2019) both based on the delta debugging algorithm (Zeller and Hildebrandt, 2002).

2002.

Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.

Improvements for AI systems

  1. Automated mixed-precision derivation for linear solvers can be implemented by using PROMISE to automate the derivation of mixed-precision mappings that would be difficult to obtain through manual analysis. This allows AI systems solving linear systems to dynamically select the lowest precision formats, such as FP16 or BF16, that satisfy user-defined accuracy requirements.

  2. Enhanced neural network training stability can be achieved by using PROMISE to identify locally optimal reduced-precision configurations in numerical software and then applying these configurations to Back Propagation kernels. This results in models that require less memory and computational time while maintaining the required gradient accuracy, as demonstrated by finding a substantial proportion of variables can be safely reduced to lower precision while preserving accuracy.

  3. Improved performance for physics simulations, such as those in HotSpot thermal simulation, can be realized by using PROMISE to explore configurations where the dominant error has shifted from local rounding to accumulated bias in the global time-step scaling. This enables the system to correctly identify when promoting variables like cell capacitance C or update terms to FP32 or BF16 yields a meaningful reduction in error.

  4. Robust particle tracking systems can benefit from using PROMISE's delta debugging algorithm across multiple precision types to ensure accuracy requirements are met during Monte Carlo localization. This systematic search identifies the locally minimal set of program locations that must retain higher precision, enabling precision reduction elsewhere, leading to faster and more reliable trajectory computations.

  5. Custom low-precision mathematical functions can be integrated into AI kernels by using FloatX extensions to emulate user-defined formats for functions like sin or exp. This allows AI systems to emulate custom floating-point formats with user-defined exponent and significand bits, allowing fine-grained control over precision levels during intermediate calculations.

Abstract

Reduced-precision arithmetic offers significant opportunities to improve performance, memory usage, and energy efficiency in numerical applications, provided that numerical accuracy is preserved. This work investigates automated precision tuning through customized floating-point formats with user-defined exponent and significand sizes, enabling the emulation of emerging low-precision formats and the exploration of non-standard precision configurations within a unified mixed-precision framework. The proposed methodology, implemented in the PROMISE precision autotuning tool, combines numerical validation with a systematic search to generate program variants that satisfy user-defined accuracy requirements. To address the computational cost of this exploration, a containerized benchmarking framework supports parallel execution across multiple algorithms and parameter configurations. The approach is evaluated on a suite of numerical programs, including linear solvers and applications from the Rodinia benchmark. Results show that a substantial proportion of variables can be safely reduced to lower precision while preserving accuracy, indicating that standard double precision is often over-provisioned. These findings highlight the potential of automated precision tuning to derive efficient mixed-precision configurations tailored to application-specific accuracy requirements.

Sources

Related papers