CUDA-L1: Improving CUDA Optimization via Contrastive Reinforcement Learning
cs.AI, cs.DC, cs.LG
Submitted: 2025-07-18
Updated: 2026-11-11
Comments: Accepted by ICLR 2026
Code: https://github.com/ornith-ai/CUDA-L1
License: http://creativecommons.org/licenses/by/4.0/
The gist: The exponential growth in demand for GPU computing resources has created an urgent need for automated CUDA optimization strategies.
Terminology
Abstract
The exponential growth in demand for GPU computing resources has created an urgent need for automated CUDA optimization strategies. While recent advances in LLMs show promise for code generation, current SOTA models achieve low success rates in improving CUDA speed. In this paper, we introduce CUDA-L1, an automated reinforcement learning framework for CUDA optimization that employs a novel contrastive RL algorithm. CUDA-L1 achieves significant performance improvements on the CUDA optimization task: trained on A100, it delivers an average speedup of x3.12 with a median speedup of x1.42 against default baselines over across all 250 CUDA kernels of KernelBench, with peak speedups reaching x120. In addition to the default baseline provided by KernelBench, CUDA-L1 demonstrates x2.77 over Torch Compile, x2.88 over Torch Compile with reduce overhead, x2.81 over CUDA Graph implementations, and x7.72 over cuDNN libraries. Furthermore, the model also demonstrates portability across different GPU architectures. Beyond these benchmark results, CUDA-L1 demonstrates several properties: it 1) discovers a variety of CUDA optimization techniques and learns to combine them strategically to achieve optimal performance; 2) uncovers fundamental principles of CUDA optimization, such as the multiplicative nature of optimizations; 3) identifies non-obvious performance bottlenecks and rejects seemingly beneficial optimizations that actually harm performance. The capabilities demonstrate that, RL can transform an initially poor-performing LLM into an effective CUDA optimizer through speedup-based reward signals alone, without human expertise or domain knowledge. This paradigm opens possibilities for automated optimization of CUDA operations, and holds promise to substantially promote GPU efficiency and alleviate the rising pressure on GPU computing resources.
Sources
- CUDA-LLM: LLMs Can Write Efficient CUDA Kernels
- The Llama 3 Herd of Models
- DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning
- GPT-4o System Card
- OpenAI o1 System Card
- Mixtral of Experts
- Evolving Deeper LLM Thinking
- DeepSeek-V3 Technical Report
- Evolution of Heuristics: Towards Efficient Automatic Algorithm Design Using Large Language Model
- s1: Simple test-time scaling
- AlphaEvolve: A coding agent for scientific and algorithmic discovery
- 2 OLMo 2 Furious
- KernelBench: Can LLMs Write Efficient GPU Kernels?
- Proximal Policy Optimization Algorithms
- DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models
- Instruction Tuning for Large Language Models: A Survey
- Gemini: A Family of Highly Capable Multimodal Models
- Gemma: Open Models Based on Gemini Research and Technology
- Reinforcement Learning Enhanced LLMs: A Survey
- SuperCoder: Assembly Program Superoptimization with Large Language Models
Related papers
- Multi-Objective Structured Pruning of LLMs for Latency and Model Size Optimization
- MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
- Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
- Embeddings for Preferences, Not Semantics
- Can We Trust Item Response Theory for AI Evaluation?
- Diagnosing Task Insensitivity in Language Agents