Instruction-Aware Transfer of GPU Implementation Crossovers: a Tensor-Product Discontinuous Galerkin Operator on GB200, H100, and A100

summary

The gist

The research compares up to six implementations of an FP64 tensor-product discontinuous Galerkin operator across GB200, H100, and A100 architectures at high polynomial degrees (p=7–1023)…

In short

The research compared six implementations of a tensor-product discontinuous Galerkin operator across GB200, H100, and A100 GPUs. It found that the fastest implementation path depends on the specific hardware; nominal performance scaling fails to predict this crossover. The authors introduced an instruction-pricing model that successfully transfers workloads between architectures based on instruction costs, proving that the optimal implementation changes with hardware.

Key concepts

Machine-Dependent Crossover
This refers to a situation where the fastest way to run a specific mathematical operation changes depending on which GPU architecture (like GB200, H100, or A100) is used. The study showed that simply looking at peak performance doesn't capture this change; the best code depends on the hardware.
Instruction-Pricing Model
This is a new mathematical model that prices every single instruction based on its specific cost for a given GPU architecture, rather than using general peak performance numbers. This pricing allows the model to accurately predict how a workload will perform on a target machine by accounting for instruction-level hardware differences.
Orthogonal Perturbation
This is a technique used to test the robustness of the instruction-pricing model. The authors restricted only some Streaming Multiprocessors (SMs) on an A100 to see if this specific change in hardware configuration could reproduce the observed performance ranking between A100 and GB200 based on memory access patterns.

Terminology used across episodes

This episode discusses

The paper

Instruction-Aware Transfer of GPU Implementation Crossovers: a Tensor-Product Discontinuous Galerkin Operator on GB200, H100, and A100 · Read on arXiv

RIKEN Center for Computational Science · The University of Tokyo

We compare up to six implementations of an FP64 tensor-product discontinuous Galerkin operator at p = 7-1023 on GB200, H100, and A100 with fixed numerics, input, and code. The fastest path is machine dependent: at p = 511, a fused-epilogue GEMM path is 5.0% faster than a fused Tensor Core kernel on GB200 and 6.9% slower on A100. Nominal-peak scaling reproduces the A100 ranking but fails on H100, where the cost of FP64 arithmetic in the profiler-defined FP64 budget depends on the instruction. We therefore price each issued instruction at its architecture-dependent cost in that budget, using a calibration-side counter profile and one derived calibration factor per kernel, with no fit to the target. A retrospective leave-one-GPU-out evaluation, holding out target timings and counters but not the model form, transfers the p >= 511 path times with 1.9% median error at the common MMA shape, and the direction of the crossover at the H100 shape. Restricting the streaming multiprocessors of one A100, with nothing fitted, reproduces the A100-to-GB200 reversal. At roduction-relevant orders the model does not resolve the fastest fused path; the result identifies the crossover mechanism.

Transcript

Introduction to the show: ident: Computer Science. The day in computer science: software, systems and theory on arXiv.

Tom: I'm Tom.

Jane: And I'm Jane, here with Lu, Meng and our in-house large language model, Lalam.

Tom: We're looking at the paper: "Instruction-Aware Transfer of GPU Implementation Crossovers".

Jane: Here's the gist of the paper. The research compares up to six implementations of an FP64 tensor-product discontinuous Galerkin operator across GB200, H100, and A100 architectures at high polynomial degrees (p=7–1023), using fixed numerics, input data, and code.

Tom: The primary finding is that the fastest implementation path is machine-dependent: at p=511, a fused-epilogue GEMM path is 5.0% faster on GB200 and 6.9% slower on A100.

Jane: Nominal-peak scaling fails to transfer this crossover, particularly for H100, where the cost of FP64 arithmetic in the profiler's FP64 budget is instruction-dependent.

Tom: To address this, the authors introduce a model that prices each issued instruction based on its architecture-dependent cost within a counter-defined FP64 budget, rather than relying on nominal machine peak performance.

Paper summary: Jane: So to wrap up the paper "Instruction-Aware Transfer of GPU Implementation Crossovers: a Tensor-Product Discontinuous Galerkin Operator on GB200, H100, and A100", what’s the real meaning for us as people who use this kind of math?

Tom: The conclusion is that the "better implementation of one operator changes between these architecture–software–stack combinations at a fixed problem." This means what's best on one machine isn't necessarily best on another.

Lu: They proved that this happens even when you keep the same mathematical operator, input data, and code base. The key is that the choice of implementation path depends on how the hardware executes instructions.

Meng: So for engineers building systems, this implies we need a way to account for these instruction-level differences when we plan our hardware choices or software stacks. It's not just about picking the highest clock speed anymore; it’s about the cost per operation.

Lalam: The model they developed is a transfer model for a workload profiled on one architecture, relying on one profiled calibration factor per kernel instance to predict performance on another without needing target timings.

Jane: It’s like having a smart predictor that tells you which machine will actually be faster based on how it handles the underlying math operations, rather than just looking at its advertised peak numbers.

Tom: So they showed that this instruction-pricing accounting correctly predicts the preferred shape of H100 for this operator. That specific shape is wider than what's seen on GB200 or A100, and it’s predicted by this model.

Lu: It shows that the implementation choice itself is tied to the hardware in a way that standard scaling doesn't capture. The paper demonstrates a mechanism for transferring performance insights across different architectures conditional on the dispatched kernel family for vendor libraries.

Meng: That’s useful because it lets us move past just comparing raw timings and start modeling what actually happens when we deploy these things at scale across different platforms.

Lalam: The study shows that this transfer is quantitative for the kernels that we compile ourselves, and it’s conditional on the dispatched kernel family for vendor libraries, which is a big piece of information for deployment scenarios.

Conclusion: Tom: So we’re wrapping up this session on that paper about the tensor product discontinuous Galerkin operator running on GB200, H100, and A100 architectures.

Jane: It boils down to this idea: how you actually write the code changes depending on which chip you put it on.

Tom: Exactly. The authors found that what’s the fastest implementation path isn't just about the peak clock speed of the machine anymore. It depends on how the hardware handles individual instructions, like FP64 arithmetic costs varying by architecture.

Lu: They looked at this across six different implementations and fixed numerics and code to see where these crossovers happened. The main finding is that nominal peak scaling doesn't actually transfer those performance advantages between these different chips.

Meng: So it means the best way to run this specific math problem isn't universal; it’s machine-dependent in a very specific, instruction-level way. What does that mean for us trying to deploy models?

Lalam: The authors propose a new model. This model prices each instruction based on its cost on that specific hardware within a defined budget instead of just using the advertised peak performance number.

Jane: That’s smart because it moves away from just looking at the sticker price of the chip and focuses on the actual work being done inside it.

Tom: And they showed that if you use this instruction pricing model, you can transfer a workload profiled on one architecture to a target architecture without needing any timing measurements for that target machine. It's a way to predict performance based on how the chips actually compute things.

Lu: They even found a specific prediction about the H100. The model correctly predicted that the preferred shape for the H100 implementation is wider than what you’d see on GB200 or A100. That shape is driven by this instruction pricing accounting.

Meng: That’s a big win because it tells us which hardware configuration, not just which chip, dictates the best software path. If we design our systems knowing this, we can make much better choices upfront.

Lalam: So the conclusion they reach is that the best way to implement one operator changes depending on how you put it on these different architecture software stacks for a fixed problem.

Jane: It’s about moving from just comparing raw timings to understanding the underlying execution cost per instruction across different hardware families.

Tom: This research suggests we need a new way of thinking about performance prediction that goes beyond just looking at the top advertised number for any given chip.

More episodes

← Home