Instruction-Aware Transfer of GPU Implementation Crossovers: a Tensor-Product Discontinuous Galerkin Operator on GB200, H100, and A100
Listen
Radio episode about this paper
Transcript
Introduction to the show: ident: Computer Science. The day in computer science: software, systems and theory on arXiv.
Tom: I'm Tom.
Jane: And I'm Jane, here with Lu, Meng and our in-house large language model, Lalam.
Tom: We're looking at the paper: "Instruction-Aware Transfer of GPU Implementation Crossovers".
Jane: Here's the gist of the paper. The research compares up to six implementations of an FP64 tensor-product discontinuous Galerkin operator across GB200, H100, and A100 architectures at high polynomial degrees (p=7–1023), using fixed numerics, input data, and code.
Tom: The primary finding is that the fastest implementation path is machine-dependent: at p=511, a fused-epilogue GEMM path is 5.0% faster on GB200 and 6.9% slower on A100.
Jane: Nominal-peak scaling fails to transfer this crossover, particularly for H100, where the cost of FP64 arithmetic in the profiler's FP64 budget is instruction-dependent.
Tom: To address this, the authors introduce a model that prices each issued instruction based on its architecture-dependent cost within a counter-defined FP64 budget, rather than relying on nominal machine peak performance.
Paper summary: Jane: So to wrap up the paper "Instruction-Aware Transfer of GPU Implementation Crossovers: a Tensor-Product Discontinuous Galerkin Operator on GB200, H100, and A100", what’s the real meaning for us as people who use this kind of math?
Tom: The conclusion is that the "better implementation of one operator changes between these architecture–software–stack combinations at a fixed problem." This means what's best on one machine isn't necessarily best on another.
Lu: They proved that this happens even when you keep the same mathematical operator, input data, and code base. The key is that the choice of implementation path depends on how the hardware executes instructions.
Meng: So for engineers building systems, this implies we need a way to account for these instruction-level differences when we plan our hardware choices or software stacks. It's not just about picking the highest clock speed anymore; it’s about the cost per operation.
Lalam: The model they developed is a transfer model for a workload profiled on one architecture, relying on one profiled calibration factor per kernel instance to predict performance on another without needing target timings.
Jane: It’s like having a smart predictor that tells you which machine will actually be faster based on how it handles the underlying math operations, rather than just looking at its advertised peak numbers.
Tom: So they showed that this instruction-pricing accounting correctly predicts the preferred shape of H100 for this operator. That specific shape is wider than what's seen on GB200 or A100, and it’s predicted by this model.
Lu: It shows that the implementation choice itself is tied to the hardware in a way that standard scaling doesn't capture. The paper demonstrates a mechanism for transferring performance insights across different architectures conditional on the dispatched kernel family for vendor libraries.
Meng: That’s useful because it lets us move past just comparing raw timings and start modeling what actually happens when we deploy these things at scale across different platforms.
Lalam: The study shows that this transfer is quantitative for the kernels that we compile ourselves, and it’s conditional on the dispatched kernel family for vendor libraries, which is a big piece of information for deployment scenarios.
Conclusion: Tom: So we’re wrapping up this session on that paper about the tensor product discontinuous Galerkin operator running on GB200, H100, and A100 architectures.
Jane: It boils down to this idea: how you actually write the code changes depending on which chip you put it on.
Tom: Exactly. The authors found that what’s the fastest implementation path isn't just about the peak clock speed of the machine anymore. It depends on how the hardware handles individual instructions, like FP64 arithmetic costs varying by architecture.
Lu: They looked at this across six different implementations and fixed numerics and code to see where these crossovers happened. The main finding is that nominal peak scaling doesn't actually transfer those performance advantages between these different chips.
Meng: So it means the best way to run this specific math problem isn't universal; it’s machine-dependent in a very specific, instruction-level way. What does that mean for us trying to deploy models?
Lalam: The authors propose a new model. This model prices each instruction based on its cost on that specific hardware within a defined budget instead of just using the advertised peak performance number.
Jane: That’s smart because it moves away from just looking at the sticker price of the chip and focuses on the actual work being done inside it.
Tom: And they showed that if you use this instruction pricing model, you can transfer a workload profiled on one architecture to a target architecture without needing any timing measurements for that target machine. It's a way to predict performance based on how the chips actually compute things.
Lu: They even found a specific prediction about the H100. The model correctly predicted that the preferred shape for the H100 implementation is wider than what you’d see on GB200 or A100. That shape is driven by this instruction pricing accounting.
Meng: That’s a big win because it tells us which hardware configuration, not just which chip, dictates the best software path. If we design our systems knowing this, we can make much better choices upfront.
Lalam: So the conclusion they reach is that the best way to implement one operator changes depending on how you put it on these different architecture software stacks for a fixed problem.
Jane: It’s about moving from just comparing raw timings to understanding the underlying execution cost per instruction across different hardware families.
Tom: This research suggests we need a new way of thinking about performance prediction that goes beyond just looking at the top advertised number for any given chip.
RIKEN Center for Computational Science · The University of Tokyo
cs.PF, cs.DC
Submitted: 2026-11-11
Updated: 2026-11-11
Comments: 35 pages, 6 figures. Includes supplementary material. Submitted to The International Journal of High Performance Computing Applications
Code: https://github.com/seiya/scale-dg-kernel-extraction
License: http://creativecommons.org/licenses/by/4.0/
Importance score: 99/100
The gist: The research compares up to six implementations of an FP64 tensor-product discontinuous Galerkin operator across GB200, H100, and A100 architectures at high polynomial degrees (p=7–1023)…
Key concepts
- Machine-Dependent Crossover
- This refers to a situation where the fastest way to run a specific mathematical operation changes depending on which GPU architecture (like GB200, H100, or A100) is used. The study showed that simply looking at peak performance doesn't capture this change; the best code depends on the hardware.
- Instruction-Pricing Model
- This is a new mathematical model that prices every single instruction based on its specific cost for a given GPU architecture, rather than using general peak performance numbers. This pricing allows the model to accurately predict how a workload will perform on a target machine by accounting for instruction-level hardware differences.
- Orthogonal Perturbation
- This is a technique used to test the robustness of the instruction-pricing model. The authors restricted only some Streaming Multiprocessors (SMs) on an A100 to see if this specific change in hardware configuration could reproduce the observed performance ranking between A100 and GB200 based on memory access patterns.
Terminology
Summary
The research compares up to six implementations of an FP64 tensor-product discontinuous Galerkin operator across GB200, H100, and A100 architectures at high polynomial degrees (p=7–1023), using fixed numerics, input data, and code. p. 0 1177/ToBeAssigned
www. com/
SAGE
Seiya Nishizawa1
, Yuta Kawai1 and Xuanzhengbo Ren2
Abstract
We compare up to six implementations of an FP64 tensor-product discontinuous Galerkin operator
at p = 7–1023 on GB200, H100, and A100 with fixed numerics, input, and code. com/
SAGE
Seiya Nishizawa1
, Yuta Kawai1 and Xuanzhengbo Ren2
Abstract
We compare up to six implementations of an FP64 tensor-product discontinuous Galerkin operator
at p = 7–1023 on GB200, H100, and A100 with fixed numerics, input, and code. The fastest path is
machine dependent: at p = 511, a fused-epilogue GEMM path is 5. The fastest path is
machine dependent: at p = 511, a fused-epilogue GEMM path is 5. 0% faster than a fused Tensor
Core kernel on GB200 and 6. 0% faster than a fused Tensor
Core kernel on GB200 and 6. 9% slower on A100.Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
To address this, the authors introduce a model that prices each issued instruction based on its architecture-dependent cost within a counter-defined FP64 budget, rather than relying on nominal machine peak performance. p. 0 Nominal-peak scaling reproduces the A100 ranking
but fails on H100, where the cost of FP64 arithmetic in the profiler-defined FP64 budget depends on
the instruction. We therefore price each issued instruction at its architecture-dependent cost in that
budget, using counter profiles from one or more calibration GPUs and one derived calibration factor
per kernel, with no fit to the target. or counters for parameter instantiation. The transfer is quantitative for the kernels that we compile
ourselves, and it is conditional on the dispatched kernel family for vendor-library kernels (Sections 6. The closer comparison for the model of Section 6 is the prediction of GPU performance for a machine on
which the workload was not measured. This prediction problem is not new.Where the paper says it
Where the paper says it
Where the paper says it
The three main contributions are:
-
Measuring a machine-dependent crossover for the fixed operator, numerics, and code base, demonstrating that nominal-peak scaling does not transfer this crossover between architecture–software–stack combinations. p. 1
Where the paper says it
By a machine-dependent crossover we mean a crossover between architecture–software-stack combinations. We show that nominal-peak scaling does not transfer this crossover (Sections 5.
-
Showing retrospectively that once the instruction-pricing form is specified, its parameters transfer a profiled workload to a target architecture without using target timings. p. 1
Where the paper says it
• We show retrospectively that, once the instruction-pricing form is specified, its parameters transfer a workload that was profiled on other architectures to a target. This transfer uses no target timings 2
-
Providing an orthogonal perturbation—restricting the active streaming multiprocessors (SMs) of one A100—which reproduces the A100-to-GB200 ranking based on matched nominal dynamic random-access memory (DRAM) bytes per active-SM cycle. p. 2
Where the paper says it
In this experiment, we restrict the active streaming multiprocessors (SMs) of one A100. The ranking crosses 1 and agrees with GB200 at matched nominal dynamic random-access memory (DRAM) bytes per active-SM cycle (Section 8).
The conclusion is that the We therefore price each issued instruction at its architecture-dependent cost in that
budget, using counter profiles from one or more calibration GPUs and one derived calibration factor
per kernel, with no fit to the target. A retrospective leave-one-GPU-out evaluation, holding out target
timings and counters but not the model form, transfers the p ≥ 511 path times with 1.better implementation of one operator changes between these architecture–software–stack combinations at a fixed problem.
Specifically, the preferred shape of H100—a wider MMA shape not present on GB200 or A100—is correctly predicted by this instruction-pricing accounting. The model is a transfer model for a workload profiled on one architecture, relying on one profiled calibration factor per kernel instance. p. 0Where the paper says it
Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.
Improvements for AI systems
-
Instruction-aware cross-architecture performance model can predict a new machine and not a new workload by pricing each issued instruction at its architecture-dependent cost in a counter defined FP64 execution budget using empirical relations established from calibration GPUs.
-
The improved system can resolve the crossover mechanism between implementation families—such as FUSED TC and GEMM FUSED—by comparing the
instruction, the library, and the epilogue fusion
choices across architectures. -
The system can accurately predict performance on unseen hardware by using a retrospective leave-one-GPU-out evaluation based on instruction pricing, achieving a median error of 1.9% at high order and up to 5.4% on A100 for the p = 511 case.
-
The model can differentiate between nominal peak scaling and machine-dependent crossovers by decomposing paths into DRAM-saturated kernels and scaling them by the ratio of HBM peaks, which reproduces the GB200-to-A100 reversal.
Abstract
We compare up to six implementations of an FP64 tensor-product discontinuous Galerkin operator at p = 7-1023 on GB200, H100, and A100 with fixed numerics, input, and code. The fastest path is machine dependent: at p = 511, a fused-epilogue GEMM path is 5.0% faster than a fused Tensor Core kernel on GB200 and 6.9% slower on A100. Nominal-peak scaling reproduces the A100 ranking but fails on H100, where the cost of FP64 arithmetic in the profiler-defined FP64 budget depends on the instruction. We therefore price each issued instruction at its architecture-dependent cost in that budget, using a calibration-side counter profile and one derived calibration factor per kernel, with no fit to the target. A retrospective leave-one-GPU-out evaluation, holding out target timings and counters but not the model form, transfers the p >= 511 path times with 1.9% median error at the common MMA shape, and the direction of the crossover at the H100 shape. Restricting the streaming multiprocessors of one A100, with nothing fitted, reproduces the A100-to-GB200 reversal. At roduction-relevant orders the model does not resolve the fastest fused path; the result identifies the crossover mechanism.
Sources
- A Few Fit Most: Improving Performance Portability of SGEMM on GPUs using Multi-Versioning
- Microbenchmark-Driven Analytical Performance Modeling Across Modern GPU Architectures
- OCCA: A unified approach to multi-threading languages
- Beyond Fast Contractions: Attenuation and Recovery of Matrix-Engine Speedups in High-Order Finite Elements
Related papers
- denet, A lightweight command-line tool for process monitoring in benchmarking and beyond
- HybridQC: Hardware-Grounded Simulation of Tightly Integrated Hybrid Quantum-Classical Systems
- TileBench: A Controlled Benchmark for Performance Evaluation and Bottleneck Diagnosis of Tile-Based Programming Models
- The M/PH 2/1 Queue: Explicit Distributions, Sojourn Times and Numerical Benchmarks
- Rolling Round-Robin Rate: Standard Heterogeneous Throughput for SPEC CPU
- Reproducible LLM Inference Benchmarking: A Sequential Isolation Protocol for Regression Testing