TIRx: A Unified Tile Primitive ML Compiler Across Heterogeneous Backends
summary
The gist
Scaling modern deep learning workloads demands coordinated placement of data and compute across device meshes, memory hierarchies…
In short
TIRx introduces a unified tile primitive ML compiler designed for heterogeneous hardware. It maps logical tensor coordinates to physical space using named axes, allowing programmers to use a single DSL for both thread-level control and high-level collective operations. This enables the compiler to automatically select optimal implementations, achieving up to 1.44x speedup on AI accelerators by overlapping communication and computation.
Key concepts
- Axe Layout
- A hardware-aware abstraction that maps logical tensor coordinates to a multi-axis physical space using named axes like thread axes or memory banks. It formalizes how data shards, replicas, and offsets relate across different devices, unifying distributed sharding and on-device tiling into one representation.
- Multi-Granularity Programming Model
- A programming approach that lets users compose logical tensors and call semantic operators at any desired level—from device operations down to individual threads. This allows programmers to mix fine-grained thread control with high-level collective operations within a single kernel definition.
- Canonicalization
- A formal procedure used by the compiler to rewrite layout definitions (like D, R, and O components) until they reach a unique form under specific gap conditions. This ensures that if two different input layouts result in the same final mapping function, their canonical forms are identical for consistent scheduling.
- Multi-Axis Physical Space
- The conceptual physical environment where data is mapped. Axe Layout defines this space using named axes that represent hardware constructs such as GPU devices, threads, and memory banks. This allows the compiler to precisely determine how logical indices translate into concrete physical locations for execution.
Terminology used across episodes
This episode discusses
- TIRx: A Unified Tile Primitive ML Compiler Across Heterogeneous Backends · Paper Radio
- DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning
- Tilus: A Tile-Level GPGPU Programming Language for Low-Precision Computation
- The Llama 3 Herd of Models · Paper Radio
- GPT-4 Technical Report
- Gemma 2: Improving Open Language Models at a Practical Size
- TileLang: A Composable Tiled Programming Model for AI Systems
- GSPMD: General and Scalable Parallelization for ML Computation Graphs
- Qwen3 Technical Report
- FlashInfer: Efficient and Customizable Attention Engine for LLM Inference Serving
- Triton-distributed: Programming Overlapping Kernels on Distributed AI Systems with the Triton Compiler
- TileLink: Generating Efficient Compute-Communication Overlapping Kernels using Tile-Centric Primitives
- Linear Layouts: Robust Code Generation of Efficient Tensor Computation Using F 2
The paper
TIRx: A Unified Tile Primitive ML Compiler Across Heterogeneous Backends · Read on arXiv
Bohan Hou, Hongyi Jin, Guanjie Wang, Jinqi Chen, Yaxing Cai, Lijie Yang, Zihao Ye
Carnegie Mellon University · Shanghai Jiao Tong University · Princeton University
Modern ML kernels evolve rapidly with new optimizations and hardware features. ML compiler abstractions must accommodate richer memory hierarchies, more forms of thread cooperation, and increasingly heterogeneous backends while retaining the control required for peak performance. We present TIRx, an ML compiler for GPUs and AI accelerators that exposes these ingredients through explicit interfaces. TIRx introduces an explicit execution-scope abstraction that captures participating hardware teams at multiple granularities. It also uses tensor layouts to map logical tensors to heterogeneous hardware spaces through named axes and shard/replica/offset semantics. Finally, it introduces tile-primitive dispatch, which lowers high-level operations to efficient implementations according to scope, layout, and backend. This design preserves native-level control while automating recurring address calculation, operand preparation, and instruction-loop generation. Our evaluation shows vendor-level performance across the latest GPUs and AI accelerators, while native-code case studies show substantial reductions in low-level implementation code.
Transcript
Introduction to the show: ident: Computer Science. The day in computer science: software, systems and theory on arXiv.
Tom: We're looking at the paper: "TIRx: A Unified Tile Primitive ML Compiler Across Heterogeneous Backends".
Jane: Here's the gist of the paper. Scaling modern deep learning workloads demands coordinated placement of data and compute across device meshes, memory hierarchies, and heterogeneous accelerators Axe Layout is a hardware-aware abstraction that maps logical tensor coordinates to a multi-axis physical space via named axes.
Tom: Axe introduces named axes to explicitly represent hardware constructs such as thread axes, memory banks in accelerators, and distributed workers.
Jane: Based on these named axes, Axe layout defines how logical coordinates of a tile map to a multi-axis physical space across GPU devices, threads, and memories.
Tom: It unifies distributed sharding and on-device memory tiling in one formal representation.
Title and authors: Tom: Let's talk about the title of this paper, "TIRx: A Unified Tile Primitive ML Compiler Across Heterogeneous Backends," and who wrote it. The core idea is that they are bringing a single compiler primitive to handle different hardware.
Jane: So, what does that mean for us practically? Is it just another way to write code or something more fundamental?
Lu: It’s about creating a unified layout abstraction called Axe Layout so that you can map logical tensor coordinates into physical space consistently across all devices.
Meng: I guess that means we don't need separate, bespoke mapping strategies for every single GPU or AI accelerator when we design our software stack.
Tom: Right. It’s about unifying how data moves and where it sits physically, whether you're talking about GPU memory or an accelerator bank.
Jane: So they are explicitly representing hardware constructs like thread axes, memory banks in accelerators, and distributed workers through these named axes in Axe Layout.
Lu: The paper proposes that you can unify tiling, sharding, replication, and offsets under this single layout abstraction for inter-device distribution and on-device layouts.
Tom: That’s a lot of concepts to pack into one formal representation to handle the complexity of heterogeneous hardware settings.
Jane: It sounds like they are trying to create a common language for defining how data should be placed before the compiler even starts generating instructions.
The paper's summary: Tom: Moving on, the summary of "TIRx" emphasizes that Axe Layout is a simple yet effective abstraction that unifies compute and data mappings across the distributed device hierarchy and heterogeneous hardware settings.
Jane: So, it’s not just about one specific piece of hardware; it’s meant to be a building block for ML compilers and frameworks at scale.
Lu: Exactly. The layout abstraction can serve as an effective building block for ML compilers and frameworks to represent and optimize workloads across various scales.
Meng: When you think about production, I worry about how this translates to actual execution cost versus just theoretical performance gains on a benchmark graph.
Tom: Well, they are showing that it works in practice by designing a multi-granularity, distribution-aware DSL that lets users compose thread-local control with collective operators in one kernel.
Jane: That means you can mix thread-local control—like what happens inside the warp—with multi-level collective operators inside one kernel definition.
Lu: Yes, this allows programmers to mix that kind of low level control with high level semantic operators in a way that's very flexible.
Tom: The compiler then reads those regions and layouts and selects the concrete implementation automatically based on what it sees.
Jane: That takes a lot of intelligence from the compiler side to figure out which specific GEMM kernel or TMA copy is best for that layout.
The paper's improvements: Tom: Now let’s look at their suggested improvements, which include tools like Canonicalize, Tile, and Slice operations within the compiler stack to help facilitate analysis and schedule dispatching.
Jane: Those sound like useful utilities for the compiler designers because they help them analyze how the layout will translate into actual hardware operations.
Lu: They are provided to make that scheduling easier by allowing the compiler to perform its own internal analysis before it decides which concrete implementation to use.
Meng: I’m interested in how those tools affect deployment complexity, though I have to admit, if they simplify the scheduling, it might help us reduce the risk of runtime errors.
Tom: The goal is that this unified approach brings performance close to hand-tuned kernels on across latest GPUs and multi-device environments.
Jane: And the results show this unified approach can bring performance up to one point four zero times speedup for multi-GPU GEMM plus ReduceScatter versus cuBLAS plus NCCL and Triton Distributed.
Lu: That’s a substantial speedup, especially when you look at distributed workloads like that on the B200 device, where they report up to one point four zero times speedup over the best baseline.
Meng: A one point four times speedup is impressive when you factor in the complexity of managing inter-device communication latency in a real production environment like ours.
Conclusion: Tom: So, wrapping up, "TIRx" provides a single mechanism to represent both perspectives in the programming model to achieve peak kernel performance with thread-level control and lowest development cost with CTA wise operators.
Jane: It offers a solid foundation for unifying layout semantics across a heterogeneous software hardware stack.
Lu: The paper concludes that it delivers competitive operator performance and practical wins for various backends, which is great because it has practical wins for different hardware targets.
Meng: For deployment, I’m just hoping that this unified approach doesn't introduce too much complexity on the operational side when we move from a research environment to something that needs to run reliably twenty-four/seven.
Tom: So the implication is that we can get high performance with fine-grained thread control without incurring a massive development cost for those complex operators.
Jane: It seems like it’s setting a solid foundation for how we think about unifying layouts in our entire deep learning software ecosystem.
Lu: Yes, it gives us a way to represent the underlying physical reality more cleanly before we try to optimize the actual hardware specifics.
Meng: I’m just waiting to see if this abstraction actually holds up when we push it into our specific production constraints.
SEGMENT: Title and authors --- <ref:2601.19092#pg1>
Tom: So we're looking at this paper, "TIRx: A Unified Tile Primitive ML Compiler Across Heterogeneous Backends," and the core idea is that they are bringing a single compiler primitive to handle different hardware.
Lu: It’s about creating a unified layout abstraction called Axe Layout so that you can map logical tensor coordinates into a physical space consistently across all devices.
More episodes
- 2505.16254-Human-AI Writing as Distributed Cognition: Reassessing Collaborative Writing Theories in the Age of LLMs
- 2506.10913-Choreographic Quick Changes: First-Class Location (Set) Polymorphism
- 2507.14812-A Black-Box Approach for Exogenous Replenishment in Online Resource Allocation
- 2507.15448-Frames over finite fields with respect to the Galois inner product and their connections to self-orthogonal and LCD codes
- 2508.21572-NewsReX: An Open-Source Multi-Framework for Neural News Recommendation
- 2509.17028-Impact of Packetization on Network Calculus Analysis
- 2510.17342-AoA Services in 5G Networks: A Framework for Real-World Implementation and Systematic Testing
- 2511.04088-Efficient and rate-optimal list-decoding in the presence of minimal feedback
- 2511.17264-Notes on Stack Machines and Quantum Stack Machines
- 2601.15318-On the closest balanced game