Deep Learning Compilers: Architecture, Optimization, and Empirical Characterization
cs.LG, cs.AR
Submitted: 2026-11-03
Updated: 2026-11-03
License: http://creativecommons.org/licenses/by/4.0/
The gist: As deep learning (DL) models become increasingly complex and hardware architectures grow more heterogeneous…
Terminology
Abstract
As deep learning (DL) models become increasingly complex and hardware architectures grow more heterogeneous, DL compilers play a critical role in optimizing computational efficiency and enabling scalable deployment. This paper presents a systematic analysis of DL compilers, covering their architectural evolution, compilation workflows, optimization techniques, and emerging research directions. We organize DL compiler architectures into frontend, middle-end, and backend stages, emphasizing the role of the middle-end in connecting high-level model representations with hardware-specific optimizations. We further examine representative DL compilers and recent optimization techniques across different compilation stages. Beyond architectural and technical analysis, we conduct an empirical evaluation of compiler behaviors to investigate the effectiveness of graph optimization, compilation overhead, and dynamic-shape handling. Experiments using PyTorch/TorchInductor on an NVIDIA RTX A6000 demonstrate that compiler benefits depend strongly on workload characteristics, with speedups of up to 8.21 times for fusion-friendly computation graphs, while other workloads exhibit limited improvements. Our results further reveal that compilation overhead can require thousands of executions to amortize, and dynamic-shape compilation reduces cumulative shape-transition costs by approximately 70% at the expense of modest steady-state performance degradation. By combining systematic architectural analysis with quantitative experimental evidence, this work provides insights into the design, optimization, and practical deployment of modern DL compilers.
Sources
- Chameleon: Adaptive Code Optimization for Expedited Deep Neural Network Compilation
- Training Deep Nets with Sublinear Memory Cost
- Meta Large Language Model Compiler: Foundation Models of Compiler Optimization
- Intel nGraph: An Intermediate Representation, Compiler, and Executor for Deep Learning
- Learning to Linearize Deep Neural Networks for Secure and Efficient Private Inference
- Combined Scheduling, Memory Allocation and Tensor Replacement for Minimizing Off-Chip Data Accesses of DNN Accelerators
- Deep Learning with Dynamic Computation Graphs
- Relay: A High-Level Compiler for Deep Learning
- Glow: Graph Lowering Compiler Techniques for Neural Networks
- Operator Fusion in XLA: Analysis and Evaluation
- AI Powered Compiler Techniques for DL Code Optimization
- Tensor Comprehensions: Framework-Agnostic High-Performance Machine Learning Abstractions
- ShapeFlow: Dynamic Shape Interpreter for TensorFlow
Related papers
- A Physics-Regulated Neural Framework for Learning 3D Grain Growth Dynamics
- FlexMoE: Learning One-for-All Multi-Budget Expert Pruning for Flexible MoE Deployment
- OptiMind: Improving Small Language Models for Optimization Modeling through Formulation Guidance and Multi-Turn Learning
- Staying in the Sweet Spot: Boost Reasoning Evolution via Responsive Rollout Difficulty Manipulation
- Examining the Impact of Data and Training Methods on Emergent and Subliminal Misalignment
- Rethinking Black-Box On-Policy Distillation: A Rubric-based Analysis