BF16 Component-Product Emulation of FP32 and FP64 GEMM on Intel AMX

arXiv:2609.04663 · cs.MS, cs.DC · Submitted 2026-09-04 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: Computer Science. The day in computer science: software, systems and theory on arXiv.

Tom: We're looking at the paper: "BF16 Component-Product Emulation of FP32 and FP64 GEMM on Intel AMX".

Jane: Here's the gist of the paper. Modern CPUs increasingly integrate high-throughput matrix engines optimized for low-precision AI workloads, while many scientific computing applications still rely on FP32 and FP64 GEMM to meet their numerical accuracy requirements <ref:2609.04663#pg2> This paper presents a CPU-oriented method based on Intel Advanced Matrix Extensions (AMX) and BF16 matrix products <ref:2609.04663#pg2> The core idea is to represent a high-precision matrix product as a controlled collection of low-precision matrix products <ref:2609.04663#pg2> Emulation Strategy The method investigates an algorithmic path across the precision barrier on CPUs by decomposing each FP32 or FP64 operand into several BF16 component matrices <ref:2609.04663#pg2>.

Tom: For FP32, it uses a three-component BF16 residual decomposition and evaluates six selected component products targeting FP32-level accuracy relative to oneMKL SGEMM <ref:2609.04663#pg2> For FP64 inputs within the supported BF16 exponent range, it applies a simplified fixed six-slice Ozaki decomposition where each retained BF16 product is produced in FP32 and then widened and accumulated in FP64 <ref:2609.04663#pg2>.

Jane: The product count is a tunable accuracy–cost control, with four settings retaining 6, 10, 15, or 21 component products to expose the accuracy–performance tradeoff relative to oneMKL DGEMM <ref:2609.04663#pg2>.

Tom: Hardware and Performance Analysis The implementation utilizes Intel Advanced Matrix Extensions (AMX) which provides a dense tile matrix engine into a general-purpose CPU <ref:2609.04663#pg2>.

Title and authors: Tom: So, we're looking at this paper now called "BF16 Component-Product Emulation of FP32 and FP64 GEMM on Intel AMX". It’s about taking those high-throughput matrix engines that modern CPUs have for low precision AI stuff and using them to actually do the heavy lifting for scientific computing that still needs high precision.

Jane: That sounds like a big bridge, Tom. So, what is the basic problem they're trying to solve here? Why can't we just use the low-precision matrix units for everything?

Tom: Well, many scientific applications need FP32 or FP64 accuracy for things like climate modeling or fluid dynamics, but these new AI matrix engines are optimized for lower precision, like BF16. This paper proposes an algorithmic bridge that uses those lower precision products to emulate the higher precision GEMM operations.

Lu: The core idea is to represent a high-precision matrix product by breaking down each FP32 or FP64 operand into several smaller BF16 component matrices and then evaluating a controlled collection of these low-precision products. This decomposition is the main way they try to get that higher precision back.

Meng: I'm thinking about what this means for deployment. If we can emulate FP32 or even FP64 using AMX, that’s huge for performance in those domains, but what’s the actual cost of this emulation?

Tom: The paper shows they use Intel Advanced Matrix Extensions, which is a tile matrix engine inside a general CPU. For FP32, they use a three component BF16 residual decomposition and evaluate six selected component products to aim for FP32 level accuracy compared to oneMKL SGEMM.

Jane: Three components and six products sounds like they are trying to find the right balance between computation time and keeping the final result accurate enough for scientific use. Are they claiming perfect elementwise identity here?

Lu: They specifically state they are not claiming bitwise or elementwise identity, which is important because it means they aren't trying to perfectly replicate a standard GEMM. Instead, they are targeting FP32-level accuracy relative to oneMKL SGEMM.

Meng: So for the FP64 inputs, what do we know about that path? Can this work for double precision data?

Tom: For FP64 inputs within the supported BF16 exponent range, they use a simplified fixed six slice Ozaki decomposition. Each BF16 product they retain is first produced in FP32, and then it gets widened and accumulated in FP64.

Jane: So for double precision data, they're doing a materialization step—converting the BF16 products to FP32 before accumulating them as FP64. That sounds like it adds some overhead on top of the low-precision computation, right?

Title and authors: Lu: Exactly. That conversion and accumulation step is what introduces the cost, but it lets them achieve that higher precision result for FP64 inputs when using AMX. They tune how many component products they keep, from six up to twenty-one, to find the sweet spot between speed and accuracy compared to oneMKL DGEMM.

Tom: And in terms of performance, they found that the AMX-FP32 path is faster than oneMKL SGEMM for all square sizes tested, but the speedup isn't smooth as you increase the matrix size.

Meng: Non-monotonic speedup? That’s a bit concerning for production. Does that mean if I scale up my problem, I might actually slow down compared to the standard library?

Jane: It means that while it works well for certain sizes, you have to watch how many component products you keep because the performance gain isn't just about scaling up the matrix size.

Lu: And they found that for AMX-FP64, low product-count variants can actually be faster than DGEMM at sufficiently large matrix orders. But as soon as you add more products to get better accuracy, the cost of storing and converting those extra FP32 results in FP64 starts to eat up the performance gain.

Tom: So it’s a clear tradeoff: you can get speed with fewer components, but if you need more precision, you have to pay for the extra accumulation and conversion work.

Jane: What about what this means for someone who just wants to run a simulation? Does this method actually give them usable FP32 or FP64 accuracy without having to use the full DGEMM routine?

Lu: It gives them a way to get close, but it’s not a free pass. You need that controlled collection of low-precision products and you have to manage how many you keep based on what your application can tolerate.

Meng: From an engineering standpoint, the implementation details matter here. They use microkernel dimensions like two fifty-six by two fifty-six by two fifty-six by thirty two by thirty two. That’s a pretty specific setup for how they are tiling things in the hardware.

Tom: And for the FP32 path, they kept those FP32 AMX accumulator tiles live across all six scheduled component products, which helped with data movement and tile load efficiency.

Jane: So that keeping the accumulators live is a key part of making it run efficiently on the CPU hardware. That’s a practical detail for someone trying to get this running in production.

Lu: And for the FP64 variants, they store each selected AMX component result as FP32, convert it to FP64, and then accumulate everything in FP64 outside of the AMX engine. That’s a very different data flow than the first path.

Title and authors: Tom: So we have two distinct paths here: one where we keep things in FP32 tiles for efficiency, and one where we materialize everything into FP64 for the final result.

Jane: This whole paper, "BF16 Component-Product Emulation of FP32 and FP64 GEMM on Intel AMX", shows how you can use hardware designed for low precision AI to bridge the gap to high precision scientific computing using decomposition.

Lu: It's a framework that uses the component-expansion principle, which they note isn't just specific to xeighty-six AMX, suggesting this idea could potentially be adapted to other CPUs that have high-throughput low-precision matrix hardware.

Meng: I’m interested in the limitations they mention. They state the attainable speedup is constrained by the current AMX tile register resources. If we need a bigger matrix than what fits in those registers, this method hits a wall and just won't scale further on that specific hardware setup.

Tom: Right, so it’s not just about the algorithm; it’s constrained by the physical resources of the specific chip you're running it on right now.

Jane: So what we have here is a technique that allows scientific computing to use low-precision matrix engines for speed while still hitting those high precision targets, provided you manage the decomposition and accept that there are tradeoffs in accuracy versus performance.

Lu: It’s less about getting perfect FP64 than it is about finding a controlled way to reconstruct a high-precision result from lower precision parts.

Meng: For deployment, it means we can use this for specific workloads where you can tune those product counts and accept the resulting accuracy profile for your needs.

Tom: So to wrap up, "BF16 Component-Product Emulation of FP32 and FP64 GEMM on Intel AMX" shows how to use AMX with BF16 products to emulate FP32 and FP64 matrix multiplication through controlled component decomposition and reconstruction.

Jane: It’s a solid piece of research because it gives a concrete, hardware-based method for people who need high precision but can't always afford the full DGEMM runtime.

Lu: The future work they suggest is extending this to rectangular or skinny matrices, which would be important for real-world applications that don't always use perfectly square matrices.

Meng: I hope we see more of this being ported to other platforms because being hardware specific like Intel AMX can limit where we can actually deploy these kinds of acceleration tricks.

Tom: That’s the main practical concern, right? So, as you go through these ideas for the next paper, keep an eye on how they expand that component-expansion idea into wider architectures.

The paper's summary: Tom: So, we're looking at this paper now called "BF16 Component-Product Emulation of FP32 and FP64 GEMM on Intel AMX". They’re basically taking these fast low-precision matrix engines meant for AI and forcing them to do the heavy lifting for high precision scientific math, like what you need for climate models.

Jane: It’s about representing a really big multiplication problem by breaking it down into smaller, lower-precision pieces and then putting those pieces back together using Intel's AMX instructions.

Lu: The core idea is that you take your FP32 or FP64 matrix product, split it into several BF16 component matrices, and evaluate a controlled selection of those products to try and get the final answer close to what you'd get from a standard high-precision operation.

Meng: So it’s an emulation strategy. They aren't just using the AI hardware for AI; they’re making it act like a more precise calculator for science. But how do you actually make that work on the chip?

Tom: For FP32, they use a three component BF16 residual decomposition and schedule six selected products to hit FP32 level accuracy when compared to the standard oneMKL SGEMM.

Jane: That means they are aiming for a very specific, known level of accuracy, not just some vague approximation. And for FP64 inputs, they use a fixed six slice Ozaki decomposition where each BF16 product is first made into FP32 and then widened into FP64 for accumulation.

Lu: The crucial part is the tuning. They test four settings where they keep six, ten, fifteen, or twenty-one component products to show exactly how much accuracy you get for every bit of performance you lose compared to the standard oneMKL DGEMM.

Meng: That sounds like a real engineering challenge because you have to manage that trade-off between how fast it runs and how close the final number is to the true answer.

Tom: And they found that for FP32, this AMX path is faster than oneMKL SGEMM across all square matrix sizes, but that speedup isn't smooth; it changes depending on how big your matrix gets.

Jane: So the implication here is that if you have a scientific workload where you can tolerate those specific accuracy trade-offs, you could get significant speedups by using this hardware instead of just running the standard double precision routines.

Lu: For FP64, they found that for very large matrix orders, keeping fewer products might actually give you a speed advantage over DGEMM, but as soon as you need more accuracy, the cost of converting those intermediate results to FP64 starts to take back the performance gain.

Meng: So it’s not a free way to get high precision; there’s always a measurable cost in terms of extra storage and conversion work.

Tom: Exactly. They also pointed out that they keep the FP32 accumulator tiles live across those six products for the FP32 path, which helps with data movement efficiency on the CPU side.

Jane: That detail about keeping those tiles live is important because it shows they optimized the data flow specifically for how Intel AMX handles its tile registers.

Lu: And they also noted that this decomposition idea isn't just for xeighty-six AMX; it’s a component expansion principle, so theoretically you could adapt this to other CPUs that have high-throughput low-precision matrix hardware.

Meng: That’s the big picture—if future hardware has better tile registers, maybe this whole framework can scale up even further.

Tom: It means the performance ceiling is currently set by how many tiles the AMX engine has available on your specific chip today.

Jane: So what this paper changes for us is showing a concrete way to use AI hardware for tasks that demand scientific precision, provided we understand exactly what accuracy we're getting versus what performance we're sacrificing.

Lu: It’s less about replacing DGEMM entirely and more about building a specific, precision-aware toolset on top of the existing low-precision matrix capabilities.

The paper's improvements: Tom: So, we’re talking about what these authors suggest as improvements for their method of emulating high precision math using low precision matrix engines on Intel AMX. They aren't just happy with the initial setup, they see a way to make it better.

Jane: Okay, so if the initial version was a proof of concept, what’s the next step in making this technique actually useful for bigger science problems?

Lu: They suggest extending these kernels and evaluations to handle more general matrix multiplication interfaces. Right now it’s mostly focused on square matrices, but they want to see how this decomposition works for rectangular or skinny matrices too.

Meng: That would be important because real-world simulations rarely use perfectly square problems; they often deal with different sized data sets.

Tom: And they also look at what happens when we move beyond the current limitations of the AMX tile registers. They explore what happens if future CPU matrix engines provide more tile registers or a larger tile capacity.

Jane: So if those future processors have bigger resources, this decomposition framework could potentially use bigger microkernels and keep more data on the chip for even better performance.

Lu: That’s the ambition; they see that component-expansion principle as something that isn't just specific to xeighty-six AMX hardware, suggesting it might work on other specialized low-precision matrix hardware as well.

Tom: So the authors are essentially laying out a roadmap for how this idea can be generalized beyond just Intel chips.

Jane: It’s interesting because it moves the discussion from a specific chip implementation to a more general mathematical representation of how these precision levels interact across different types of processors.

Meng: From an engineering standpoint, that means if we build our own accelerator hardware later, we have this existing idea they can use as a blueprint for how to handle precision mismatches.

Lu: Exactly. And they are also focusing on making sure the method remains robust when dealing with inputs outside the normal BF16 range, like handling potential overflow or underflow in FP64 inputs.

Tom: So they are trying to make it more reliable for real-world scientific data, not just perfect test cases.

Jane: It’s a step toward making these emulation techniques practical for actual high-performance computing where data isn't always perfectly behaved.

Conclusion: Tom: So we’re wrapping up this look at "BF16 Component-Product Emulation of FP32 and FP64 GEMM on Intel AMX". Basically, they showed how you can use low precision matrix engines to get high precision results for scientific computing.

Jane: It’s a solid framework because it shows a concrete way to bridge the gap between fast AI hardware and demanding numerical accuracy requirements.

Lu: The core finding is that you get this controlled collection of low-precision products, and by tuning how many you keep, you can find an accuracy cost tradeoff for FP64 inputs.

Meng: For production, it means we can use this to tailor the precision to the task at hand instead of just running a single standard routine.

Tom: Right. They found that for FP32, the AMX path is faster than its baseline SGEMM across all square sizes tested, even if that speedup isn't smooth as your matrix gets bigger.

Jane: That means for certain workloads, you might actually see a speed boost by using this hardware instead of the standard library code.

Lu: And for FP64 inputs, they found that low product counts can actually beat DGEMM at large matrix orders, but adding more products just increases the overhead from materialization and conversion to FP64.

Meng: So it’s a clear constraint: performance gains come with a direct cost in terms of extra work outside of the main AI engine.

Tom: The method relies heavily on Intel AMX instructions, specifically its BF16 tile instructions, to manage this emulation.

Jane: It’s important to remember that this specific setup is constrained by the current tile register resources on the chip you are using right now.

Lu: And they pointed out that for future work, they plan to extend this framework to rectangular or skinny matrices, which would be crucial for real-world applications that don't always use perfectly square problems.

Meng: If future CPU matrix engines provide more tile registers or bigger capacity, this whole decomposition idea could scale up even further on new hardware platforms.

Tom: So the main thing is that this component expansion principle isn't just locked into xeighty-six AMX, it’s a concept we can adapt to other high-throughput low precision matrix hardware.

Jane: It’s a good reminder that the potential for using AI hardware in scientific computing is bigger than we initially thought.

Lu: I think what this paper really changes for the field is showing a generalized way to use low precision products as building blocks for higher precision computation, regardless of the specific chip architecture.

Meng: I just hope we see more of this being ported to other platforms because being hardware specific like Intel AMX can limit where we can actually deploy these kinds of acceleration tricks.

Tom: Well said. So that’s the summary for "BF16 Component-Product Emulation of FP32 and FP64 GEMM on Intel AMX".

Jane: It’s a solid piece of research showing how you can use low precision matrix engines to emulate high precision math through decomposition.

Lu: This is less about perfect emulation and more about finding a controlled way to reconstruct high precision results from lower precision parts.

Meng: It’s a framework that lets us tailor the precision to the task at hand while managing the performance cost.

Maginfra Co., Ltd.

cs.MS, cs.DC

Submitted: 2026-09-04

Updated: 2026-11-11

License: http://creativecommons.org/licenses/by/4.0/

Importance score: 83/100

The gist: Modern CPUs increasingly integrate high-throughput matrix engines optimized for low-precision AI workloads…

Key concepts

BF16 Component Decomposition
This technique breaks down an FP32 or FP64 matrix multiplication into several smaller, lower-precision BF16 matrix products. For example, an FP32 operation is represented by a combination of three BF16 components. This decomposition allows the high-precision calculation to be approximated using the faster low-precision hardware available in AMX.
AMX Tile Engine
Intel Advanced Matrix Extensions (AMX) provides a dense tile matrix engine within a general-purpose CPU. The paper leverages this engine, specifically its BF16 instructions, to perform the core low-precision matrix multiplications efficiently. This hardware is key to achieving speedups over standard software implementations.
Accuracy/Cost Tradeoff
The method allows users to control the accuracy of the emulation by choosing how many BF16 component products are retained in the calculation. For FP64, increasing this number improves agreement with DGEMM accuracy but increases computational cost. This lets engineers select a schedule that balances required numerical precision against performance needs.
Operand Reuse Schedule
For the AMX-FP32 path, a specific scheduling strategy is used to keep the FP32 AMX accumulator tiles active across multiple scheduled component products. This reuse of data minimizes costly data movement and tile loading, significantly improving efficiency for large matrix operations.

Terminology

Summary

Modern CPUs increasingly integrate high-throughput matrix engines optimized for low-precision AI workloads, while many scientific computing applications still rely on FP32 and FP64 GEMM to meet their numerical accuracy requirements <ref:2609. p. 0

Where the paper says it

Abstract Modern CPUs increasingly integrate high-throughput matrix engines optimized for lowprecision AI workloads, while many scientific computing applications still rely on FP32 and FP64 GEMM to meet their numerical accuracy requirements. This mismatch motivates an algorithmic bridge that uses low-precision matrix products to emulate higher-precision GEMM.

04663#pg2> This paper presents a CPU-oriented method based on Intel Advanced Matrix Extensions (AMX) and BF16 matrix products <ref:2609. p. 0
Where the paper says it

This mismatch motivates an algorithmic bridge that uses low-precision matrix products to emulate higher-precision GEMM. This paper presents a CPU-oriented method based on Intel Advanced Matrix Extensions (AMX) and BF16 matrix products.

04663#pg2> The core idea is to represent a high-precision matrix product as a controlled collection of low-precision matrix products <ref:2609. p. 1
Where the paper says it

The goal of this paper is to investigate an algorithmic path across this precision barrier on CPUs. The core idea is to represent a high-precision matrix product as a controlled collection of low-precision matrix products: each FP32 or FP64 operand is decomposed into several BF16 component matrices, selected component products are evaluated by a fast tile engine, and the partial results are reconstructed in a wider format.

p. 1
Where the paper says it

The goal of this paper is to investigate an algorithmic path across this precision barrier on CPUs. The core idea is to represent a high-precision matrix product as a controlled collection of low-precision matrix products: each FP32 or FP64 operand is decomposed into several BF16 component matrices, selected component products are evaluated by a fast tile engine, and the partial results are reconstructed in a wider format.

04663#pg2>

Emulation Strategy

The method investigates an algorithmic path across the precision barrier on CPUs by decomposing each FP32 or FP64 operand into several BF16 component matrices <ref:2609.04663#pg2>. For FP32, it uses a three-component BF16 residual decomposition and evaluates six selected component products targeting FP32-level accuracy relative to oneMKL SGEMM <ref:2609. p. 0

Where the paper says it

For FP32, each operand is decomposed into three BF16 components and six selected component products are evaluated, targeting FP32-level accuracy relative to oneMKL SGEMM without claiming elementwise or bitwise identity. For FP64 inputs within the supported BF16 exponent range, the method uses a simplified fixed six-slice Ozaki decomposition.

04663#pg2> For FP64 inputs within the supported BF16 exponent range, it applies a simplified fixed six-slice Ozaki decomposition where each retained BF16 product is produced in FP32 and then widened and accumulated in FP64 <ref:2609. p. 0
Where the paper says it

For FP64 inputs within the supported BF16 exponent range, the method uses a simplified fixed six-slice Ozaki decomposition. Each retained BF16 product is first produced in FP32, then widened and accumulated in FP64.

04663#pg2>. The product count is a tunable accuracy–cost control, with four settings retaining 6, 10, 15, or 21 component products to expose the accuracy–performance tradeoff relative to oneMKL DGEMM <ref:2609. p. 0
Where the paper says it

Each retained BF16 product is first produced in FP32, then widened and accumulated in FP64. Four product-count settings retain 6, 10, 15, or 21 component products, exposing the accuracy–performance tradeoff relative to oneMKL DGEMM.

04663#pg2>.

Hardware and Performance Analysis

The implementation utilizes Intel Advanced Matrix Extensions (AMX) which provides a dense tile matrix engine into a general-purpose CPU <ref:2609.04663#pg2>. The AMX-FP32 path exploits the shared exponent range of BF16 and FP32 through a three-component BF16 residual decomposition and the six-product schedule suggested by Henry et al. p. 1

Where the paper says it

This paper presents a BF16-based method for emulating FP32 and FP64 GEMM using Intel AMX. The AMX-FP32 path exploits the shared exponent range of BF16 and FP32 through a three-component BF16 residual decomposition and the six-product schedule suggested by Henry et al.

<ref:2609.04663#pg2> The AMX-FP32 path is faster than oneMKL SGEMM for all tested square sizes, but the speedup is non-monotonic with matrix size <ref:2609. p. 0
Where the paper says it

The implementation combines precomputed packed component buffers, VNNI-packed B panels, and an FP32 tile-resident operand-reuse schedule. On the tested square matrices, AMX-FP32 exceeds oneMKL SGEMM throughput.

04663#pg2>. For AMX-FP64, low-product-count variants can exceed DGEMM at sufficiently large orders, while retaining more products improves accuracy at additional cost <ref:2609. p. 0
Where the paper says it

On the tested square matrices, AMX-FP32 exceeds oneMKL SGEMM throughput. For AMX-FP64, low-product-count variants can exceed DGEMM at sufficiently large orders, while retaining more products improves accuracy at additional cost.

04663#pg2>.

Accuracy and Tradeoff

The attained accuracy is measured experimentally against the corresponding oneMKL GEMM result <ref:2609.04663#pg2> For AMX-FP32, the comparison determines whether the six-product AMX-FP32 kernel provides FP32-level accuracy relative to oneMKL SGEMM <ref:2609. p. 1

Where the paper says it

The AMX-FP32 path exploits the shared exponent range of BF16 and FP32 through a three-component BF16 residual decomposition and the six-product schedule suggested by Henry et al. [4], and evaluates whether the resulting kernel provides FP32-level accuracy relative to oneMKL SGEMM.

04663#pg2>. For AMX-FP64 variants, increasing the number of retained BF16 component products sharply reduces both Frobenius relative error and the GEMM-scaled componentwise error <ref:2609. p. 8
Where the paper says it

This choice differs from a full Ozaki configuration, where the decomposition depth and scaling parameters are chosen from error bounds so that the final result satisfies a prescribed FP64 accuracy target. Here the goal is instead to expose the accuracy/performance tradeoff available on AMX: each operand is split into six BF16 slices, and the number of retained component products is varied.

04663#pg2> The results show that retaining more component products improves agreement with the DGEMM reference <ref:2609. p. 1
Where the paper says it

It fixes six BF16 slices per operand, reconstructs each selected component product in FP64, and varies the number of retained cross-products. The product count is a tunable accuracy–cost control: retaining more products typically improves agreement with oneMKL DGEMM but reduces the performance headroom available for acceleration.

04663#pg2>.

Implementation Details

The implementation follows a conventional layered structure of a high-performance GEMM, using microkernel dimensions of (BM, BN, BK, TM, TN) = (256, 256, 256, 32, 32) <ref:2609.04663#pg2>. The AMX-FP32 path keeps the FP32 AMX accumulator tiles live across the six scheduled component products to improve data movement and tile-load efficiency <ref:2609. p. 6

Where the paper says it

Packed BF16 operand tile Packed BF16 operand tile AMX source tile BF16 AMX source tile BF16 tdpbf16ps BF16 × BF16 → FP32 accumulate AMX destination tile FP32 accumulator Stored FP32 tile result Post-AMX reconstruction AMX tile state inside one CPU core Figure 1: AMX-BF16 execution primitive. TILELOADD explicitly loads BF16 source tiles, TDPBF16PS accumulates their dot products into an FP32 destination tile, and TILESTORED writes the accumulated tile to memory.

04663#pg2>. In contrast, the AMX-FP64 variants store each selected AMX component result as FP32, exactly embed it into FP64, and then accumulate each selected product in FP64 <ref:2609. p. 1
Where the paper says it

The core idea is to represent a high-precision matrix product as a controlled collection of low-precision matrix products: each FP32 or FP64 operand is decomposed into several BF16 component matrices, selected component products are evaluated by a fast tile engine, and the partial results are reconstructed in a wider format. Intel Advanced Matrix Extensions (AMX) are the implementation vehicle studied here because they bring a dense tile matrix engine into a generalpurpose CPU.

04663#pg2>.

Conclusion

The paper presents a framework that uses Intel AMX, specifically its AMX-BF16 tile instructions, to emulate FP32 and FP64 matrix multiplication <ref:2609.04663#pg2> The AMX-FP32 path combines a three-component BF16 residual decomposition with the six-product schedule suggested by Henry et al. p. 1

Where the paper says it

This paper presents a BF16-based method for emulating FP32 and FP64 GEMM using Intel AMX. The AMX-FP32 path exploits the shared exponent range of BF16 and FP32 through a three-component BF16 residual decomposition and the six-product schedule suggested by Henry et al.

p. 1
Where the paper says it

This paper presents a BF16-based method for emulating FP32 and FP64 GEMM using Intel AMX. The AMX-FP32 path exploits the shared exponent range of BF16 and FP32 through a three-component BF16 residual decomposition and the six-product schedule suggested by Henry et al.

and an operand-reuse schedule that keeps FP32 AMX accumulator tiles resident and reduces tileload traffic <ref:2609. p. 0
Where the paper says it

The implementation combines precomputed packed component buffers, VNNI-packed B panels, and an FP32 tile-resident operand-reuse schedule. On the tested square matrices, AMX-FP32 exceeds oneMKL SGEMM throughput.

p. 0
Where the paper says it

The implementation combines precomputed packed component buffers, VNNI-packed B panels, and an FP32 tile-resident operand-reuse schedule. On the tested square matrices, AMX-FP32 exceeds oneMKL SGEMM throughput.

04663#pg2> For BF16-range FP64 inputs, the AMX-FP64 path uses a simplified fixed six-slice Ozaki decomposition where each selected BF16 component product is materialized, converted from FP32, and accumulated in FP64 <ref:2609. p. 0
Where the paper says it

For FP64 inputs within the supported BF16 exponent range, the method uses a simplified fixed six-slice Ozaki decomposition. Each retained BF16 product is first produced in FP32, then widened and accumulated in FP64.

p. 0
Where the paper says it

For FP64 inputs within the supported BF16 exponent range, the method uses a simplified fixed six-slice Ozaki decomposition. Each retained BF16 product is first produced in FP32, then widened and accumulated in FP64.

04663#pg2>. The attainable speedup is also constrained by the current AMX tile-register resources <ref:2609. p. 5
Where the paper says it

Equation (4) motivates the three implementation principles used in this paper: minimize insignificant component products, fuse decomposition with the packing path already required by blocked GEMM, and reuse tile-resident operands across multiple products. 3 CPU Matrix Engines and Intel AMX Recent Xeon processors incorporate Intel AMX, an x86 instruction-set extension that adds a CPU-resident tile-register matrix facility [3].

p. 5
Where the paper says it

Equation (4) motivates the three implementation principles used in this paper: minimize insignificant component products, fuse decomposition with the packing path already required by blocked GEMM, and reuse tile-resident operands across multiple products. 3 CPU Matrix Engines and Intel AMX Recent Xeon processors incorporate Intel AMX, an x86 instruction-set extension that adds a CPU-resident tile-register matrix facility [3].

04663#pg2>. The component-expansion principle is also not specific to x86 AMX, and the method can be adapted to CPUs that integrate high-throughput low-precision matrix hardware <ref:2609. p. 0
Where the paper says it

Abstract Modern CPUs increasingly integrate high-throughput matrix engines optimized for lowprecision AI workloads, while many scientific computing applications still rely on FP32 and FP64 GEMM to meet their numerical accuracy requirements. This mismatch motivates an algorithmic bridge that uses low-precision matrix products to emulate higher-precision GEMM.

p. 0
Where the paper says it

Abstract Modern CPUs increasingly integrate high-throughput matrix engines optimized for lowprecision AI workloads, while many scientific computing applications still rely on FP32 and FP64 GEMM to meet their numerical accuracy requirements. This mismatch motivates an algorithmic bridge that uses low-precision matrix products to emulate higher-precision GEMM.

p. 0
Where the paper says it

Abstract Modern CPUs increasingly integrate high-throughput matrix engines optimized for lowprecision AI workloads, while many scientific computing applications still rely on FP32 and FP64 GEMM to meet their numerical accuracy requirements. This mismatch motivates an algorithmic bridge that uses low-precision matrix products to emulate higher-precision GEMM.

04663#pg2>.

The paper is 2609. p. 0

Where the paper says it

In machine learning, GEMM is the dominant dense linear-algebra ∗Correspondence to: liujerry06st@gmail. com 1 arXiv:2609.

04663. The single most informative sentence about the paper's subject and finding is: Modern CPUs increasingly integrate high-throughput matrix engines optimized for low-precision AI workloads, while many scientific computing applications still rely on FP32 and FP64 GEMM to meet their numerical accuracy requirements <ref:2609.04663#pg2>.

Pruning redundant reasoning tokens

The paper presents a CPU-oriented method based on Intel Advanced Matrix Extensions (AMX) and BF16 matrix products <ref:2609.04663#pg2>. The core idea is to represent a high-precision matrix product as a controlled collection of low-precision matrix products <ref:2609.04663#pg2>. This approach is motivated by the mismatch between AI-oriented low-precision throughput and the numerical accuracy requirements of scientific applications <ref:2609. p. 0

Where the paper says it

Abstract Modern CPUs increasingly integrate high-throughput matrix engines optimized for lowprecision AI workloads, while many scientific computing applications still rely on FP32 and FP64 GEMM to meet their numerical accuracy requirements. This mismatch motivates an algorithmic bridge that uses low-precision matrix products to emulate higher-precision GEMM.

04663#pg2>.

How the training signal is built

The AMX-FP32 path exploits the shared exponent range of BF16 and FP32 through a three-component BF16 residual decomposition and the six-product schedule suggested by Henry et al. <ref:2609.04663#pg2>. This decomposition uses three BF16 components per operand, defining residuals A(r) and B(r) <ref:2609. p. 7

Where the paper says it

2 AMX-FP32 Emulation For FP32, BF16 and FP32 share the normal exponent range. A direct residual decomposition therefore uses three BF16 components per operand.

04663#pg2>. The resulting six selected component products are defined by the triangular selection Sd = (i, j: 0 ≤ i < sA, 0 ≤ j < sB, i + j < d), where d=3 recovers the six-product structure of Henry et al. p. 7
Where the paper says it

It nevertheless gives the standard triangular truncation used to trade componentproduct count against accuracy [4, 21]. For AMX-FP32, d = 3 recovers the six-product structure of Henry et al.

[4] <ref:2609. p. 0
Where the paper says it

In machine learning, GEMM is the dominant dense linear-algebra ∗Correspondence to: liujerry06st@gmail. com 1 arXiv:2609.

04663#pg2>.

Where the gains come from

The AMX-FP32 path targets FP32-level accuracy relative to oneMKL SGEMM rather than bitwise or elementwise identity <ref:2609.04663#pg2>. For AMX-FP64, retaining more component products improves agreement with the DGEMM reference <ref:2609. p. 1

Where the paper says it

It fixes six BF16 slices per operand, reconstructs each selected component product in FP64, and varies the number of retained cross-products. The product count is a tunable accuracy–cost control: retaining more products typically improves agreement with oneMKL DGEMM but reduces the performance headroom available for acceleration.

04663#pg2>. The performance headroom is limited because each retained FP64 component product must be materialized, converted to FP64, and accumulated outside AMX <ref:2609. p. 0
Where the paper says it

Each retained BF16 product is first produced in FP32, then widened and accumulated in FP64. Four product-count settings retain 6, 10, 15, or 21 component products, exposing the accuracy–performance tradeoff relative to oneMKL DGEMM.

04663#pg2>.

Numerical Accuracy Considerations

The attained accuracy is measured experimentally against the corresponding oneMKL GEMM result <ref:2609.04663#pg2>. The AMX-FP32 path remains in the expected single-precision range, with the Frobenius relative error staying on the order of 10−7 <ref:2609. p. 5

Where the paper says it

For AMX-FP64, no single pass threshold is imposed. The six-slice variants are compared with CD to determine how accuracy improves as the number of BF16 component GEMMs increases from 6 to 21 and how much performance remains relative to DGEMM.

04663#pg2>. For AMX-FP64 variants, increasing the number of retained BF16 component products sharply reduces both metrics <ref:2609. p. 1
Where the paper says it

It fixes six BF16 slices per operand, reconstructs each selected component product in FP64, and varies the number of retained cross-products. The product count is a tunable accuracy–cost control: retaining more products typically improves agreement with oneMKL DGEMM but reduces the performance headroom available for acceleration.

04663#pg2>. The product count serves as a controllable accuracy–cost parameter of the AMX-FP64 design <ref:2609. p. 0
Where the paper says it

For AMX-FP64, low-product-count variants can exceed DGEMM at sufficiently large orders, while retaining more products improves accuracy at additional cost. Keywords: Intel AMX, BF16, FP32 and FP64 GEMM, mixed-precision, high-performance computing 1 Introduction The rapid expansion of artificial intelligence has reshaped the design of modern compute hardware.

04663#pg2>.

GEMM Performance

The AMX-FP32 path is faster than oneMKL SGEMM for all tested square sizes, but the speedup is non-monotonic with matrix size <ref:2609.04663#pg2>. For AMX-FP64, performance gains over DGEMM are available only for sufficiently truncated schedules because the additional store, conversion, and FP64 accumulation costs consume the limited AMX-BF16 headroom <ref:2609. p. 5

Where the paper says it

Consequently, BF16 decomposition can represent the FP32 path directly and can support the BF16-range FP64 path without per-panel scaling in the supported domain. An FP16-based expansion would generally require additional scaling, scale metadata, and reconstruction operations; those costs would consume the limited performance headroom of a multi-product emulation method.

04663#pg2>. The runtime breakdown shows that for AMX-FP32, the preprocessing share falls from one third of the total runtime at N = 256 to less than 4% at N = 32768 <ref:2609.04663#pg2>.

Runtime Breakdown

For AMX-FP32, the FP32 AMX accumulator tiles remain live across the scheduled products <ref:2609.04663#pg2> For AMX-FP64, each selected BF16 matrix multiplication is materialized, converted to FP64, and accumulated in FP64 <ref:2609. p. 4

Where the paper says it

The input-domain symbol Dt constrains the operands, rather than the reconstructed FP64 output. Thus, even when A and B belong to F B 64, C need not belong to F B 64: component results are widened and accumulated in FP64, and the output is never converted back to BF16.

04663#pg2>. This distinction means that the AMX-FP32 path keeps the AMX FP32 accumulator tiles live across the six scheduled component products while the AMX-FP64 variants store each selected AMX BF16 product, convert it to FP64, and accumulate outside AMX <ref:2609. p. 4
Where the paper says it

Finite FP64 values x /∈ F B 64 are likewise outside the supported numerical domain of the implementation studied here. 2 Problem Definition and Performance Model This work considers only the matrix product C = AB, A ∈ Dm×k t , B ∈ Dk×n t , C ∈ F m×n t , (3) where Ft = F32 and Dt = F32 for the AMX-FP32 path, while Ft = F64 and Dt = F B 64 for the BF16-range AMX-FP64 path.

04663#pg2>.

Future Work

Future work will extend the kernels and evaluation to rectangular, skinny, or more general matrix-multiplication interfaces <ref:2609.04663#pg2> The attainable speedup is also constrained by the current AMX tile-register resources <ref:2609. p. 5

Where the paper says it

Equation (4) motivates the three implementation principles used in this paper: minimize insignificant component products, fuse decomposition with the packing path already required by blocked GEMM, and reuse tile-resident operands across multiple products. 3 CPU Matrix Engines and Intel AMX Recent Xeon processors incorporate Intel AMX, an x86 instruction-set extension that adds a CPU-resident tile-register matrix facility [3].

p. 5
Where the paper says it

Equation (4) motivates the three implementation principles used in this paper: minimize insignificant component products, fuse decomposition with the packing path already required by blocked GEMM, and reuse tile-resident operands across multiple products. 3 CPU Matrix Engines and Intel AMX Recent Xeon processors incorporate Intel AMX, an x86 instruction-set extension that adds a CPU-resident tile-register matrix facility [3].

04663#pg2>. If future CPU matrix engines provide more tile registers, larger tile capacity, or both, the same decomposition framework could use larger microkernels and retain more operands or accumulators on chip <ref:2609. p. 5
Where the paper says it

Equation (4) motivates the three implementation principles used in this paper: minimize insignificant component products, fuse decomposition with the packing path already required by blocked GEMM, and reuse tile-resident operands across multiple products. 3 CPU Matrix Engines and Intel AMX Recent Xeon processors incorporate Intel AMX, an x86 instruction-set extension that adds a CPU-resident tile-register matrix facility [3].

04663#pg2>. The component-expansion principle is also not specific to x86 AMX, and the method can be adapted to CPUs that integrate high-throughput low-precision matrix hardware <ref:2609.04663#pg2>.

Future work will extend the kernels and evaluation to rectangular, skinny, or more general matrix-multiplication interfaces <ref:2609.04663#pg2> The attainable speedup is also constrained by the current AMX tile-register resources <ref:2609.04663#pg2> If future CPU matrix engines provide more tile registers, larger tile capacity, or both, the same decomposition framework could use larger microkernels and retain more operands or accumulators on chip <ref:2609. p. 5

Where the paper says it

Equation (4) motivates the three implementation principles used in this paper: minimize insignificant component products, fuse decomposition with the packing path already required by blocked GEMM, and reuse tile-resident operands across multiple products. 3 CPU Matrix Engines and Intel AMX Recent Xeon processors incorporate Intel AMX, an x86 instruction-set extension that adds a CPU-resident tile-register matrix facility [3].

04663#pg2>. The component-expansion principle is also not specific to x86 AMX, and the method can be adapted to CPUs that integrate high-throughput low-precision matrix hardware <ref:2609.04663#pg2>.

The paper presents a framework that uses Intel AMX, specifically its AMX-BF16 tile instructions, to emulate FP32 and FP64 matrix multiplication <ref:2609.04663#pg2> The AMX-FP32 path combines a three-component BF16 residual decomposition with the six-product schedule suggested by Henry et al. and an operand-reuse schedule that keeps FP32 AMX accumulator tiles resident and reduces tileload traffic <ref:2609.04663#pg2> For BF16-range FP64 inputs, the AMX-FP64 path uses a simplified fixed six-slice Ozaki decomposition where each selected BF16 component product is materialized, converted from FP32, and accumulated in FP64 <ref:2609.04663#pg2>. The attainable speedup is also constrained by the current AMX tile-register resources <ref:2609.04663#pg2> The component-expansion principle is also not specific to x86 AMX, and the method can be adapted to CPUs that integrate high-throughput low-precision matrix hardware <ref:2609. p. 0

Where the paper says it

Abstract Modern CPUs increasingly integrate high-throughput matrix engines optimized for lowprecision AI workloads, while many scientific computing applications still rely on FP32 and FP64 GEMM to meet their numerical accuracy requirements. This mismatch motivates an algorithmic bridge that uses low-precision matrix products to emulate higher-precision GEMM.

04663#pg2>.

Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.

Improvements for AI systems

  1. Precision Emulation for Scientific Computing Systems can emulate FP32 and FP64 GEMM using Intel AMX and BF16 matrix products to meet high-precision numerical accuracy requirements on general-purpose CPUs. This allows scientific applications like computational fluid dynamics or climate simulation to utilize the throughput of low-precision matrix engines while maintaining necessary numerical stability.

  2. AMX FP32 emulation can achieve FP32 level accuracy relative to oneMKL SGEMM by using a three component BF16 residual decomposition and a six product schedule with an operand reuse schedule that keeps FP32 AMX accumulator tiles resident, which reduces tile load traffic. This provides a plausible performance opportunity over native SGEMM for large matrices where the FP32 operand reuse is effective.

  3. For BF16-range FP64 inputs, the AMX FP64 path can expose an accuracy cost tradeoff by varying the number of retained BF16 component products from 6 to 21, allowing engineers to choose a schedule that balances performance against DGEMM accuracy. This enables the system to operate in a precision-aware regime rather than claiming complete FP64 or DGEMM-equivalent accuracy universally.

  4. The system can be designed with scale aware decomposition and fallback handling for BF16-range FP64 inputs, extending the supported input domain beyond the normal BF16 range and addressing cases involving overflow or underflow. This future extension would allow the framework to handle a broader set of HPC data types reliably.

Abstract

Modern CPUs increasingly integrate high-throughput matrix engines optimized for low-precision AI workloads, while many scientific computing applications still rely on FP32 and FP64 GEMM to meet their numerical accuracy requirements. This paper presents a method that combines BF16 matrix products on Intel Advanced Matrix Extensions (AMX) to emulate real square FP32 and FP64 GEMM. For FP32 GEMM, each element in both input matrices is decomposed into three BF16 components, and six selected component products are evaluated to target FP32-level accuracy comparable to oneMKL SGEMM. For FP64 inputs within the supported BF16 exponent range, the method uses a simplified fixed six-slice Ozaki decomposition. Each retained BF16 product is first produced in FP32, then widened and accumulated in FP64. Four product-count settings retain 6, 10, 15, or 21 component products, exposing the accuracy-performance tradeoff relative to oneMKL DGEMM. The FP32 schedule reuses component data and retains partial sums in AMX tiles. AMX-FP32 outperforms oneMKL SGEMM at tested orders N at least4096 on 64 threads and also achieves gains for smaller matrices at lower thread counts. AMX-FP64 variants retaining fewer component products can outperform oneMKL DGEMM at sufficiently large orders; more products improve accuracy at additional cost.

Sources

Related papers