ModelBlaster: Closing the Performance Gap through Learning-Augmented Agents Across Models, Kernels and Hardware Backends
cs.LG
Submitted: 2026-11-03
Updated: 2026-11-03
Comments: 10 pages of main content and 23 pages of appendices
Code: https://github.com/ucb-bar/ModelBlaster
License: http://creativecommons.org/licenses/by-nc-nd/4.0/
The gist: Optimizing accelerator code across hardware architectures is difficult.
Terminology
Abstract
Optimizing accelerator code across hardware architectures is difficult. ML compilers search predefined spaces of schedules and micro-kernels, while LLM-guided optimization can explore alternative algorithms, operator decompositions, and hardware-specific techniques. However, existing LLM agents typically target a single programming interface, toolchain, or accelerator family, focus on kernel-level optimization without model-level transforms, and offer limited reuse of experience across hardware targets. We present ModelBlaster, a Memory-Augmented In-Context Reinforcement Learning (MAIC-RL) framework for optimizing data-parallel kernels and whole models across heterogeneous SIMD and SIMT accelerators. Its mutable policy is a persistent knowledge base updated at inference time by Actor-Critic Textual Gradient Descent (AC-TGD). The knowledge base maps performance states to actions, outcomes, and constraints, enabling experience reuse across kernels, models, and hardware targets. A model-level optimizer uses kernel-level knowledge to inform whole-model decisions. The same optimization loop and knowledge schema support CUDA on NVIDIA GPUs, OpenCL on Qualcomm GPUs, and RVV intrinsics on RISC-V vector cores, with only lowering and measurement layers specific to each target. Compared with PyTorch, ModelBlaster achieves geometric-mean speedups of 1.85x, 5.58x, and 5.18x on KernelBench Levels 1, 2, and 3, and 1.69x on SOL-ExecBench Levels 1 and 2 on a B200 GPU. On Qualcomm Adreno GPUs, it achieves 5.8x over the vendor SDK and 1.7x over Apache TVM; on RISC-V vector cores, it achieves 2.7x over ExecuTorch with XNNPACK across 70 KernelBench problems.
Sources
- How Much of a Real Workload Can LLM-Generated GPU Kernels Actually Reach?
- GEPA: Reflective Prompt Evolution Can Outperform Reinforcement Learning
- Accelerating GenAI Workloads by Enabling RISC-V Microkernel Support in IREE
- CodeEvolve: an open source evolutionary coding agent for algorithmic discovery and optimization
- KernelBrain: Coarse-to-Fine, Budget-Aware Search for Agentic GPU Kernel Optimization
- Evolution of Kernels: Automated RISC-V Kernel Optimization with Large Language Models
- AVO: Agentic Variation Operators for Autonomous Evolutionary Search
- CUDA-LLM: LLMs Can Write Efficient CUDA Kernels
- MPK: A Compiler and Runtime for Mega-Kernelizing Tensor Programs
- FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness
- Contour: Extensible Mapping Surfaces for Agentic Accelerator Kernel Generation
- Pie: A Programmable Serving System for Emerging LLM Applications
- Tuning LLM-based Code Optimization via Meta-Prompting: An Industrial Perspective
- EvoEngineer: Mastering Automated CUDA Kernel Code Evolution with Large Language Models
- Optimal Kernel Orchestration for Tensor Programs with Korch
- FlashFuser: Expanding the Scale of Kernel Fusion for Compute-Intensive Operators via Inter-Core Connection
- AutoKernel: Autonomous GPU Kernel Optimization via Iterative Agent-Driven Search
- Tile-Level Activation Overlap for Efficient LLM Inference
- Towards Robust Agentic CUDA Kernel Benchmarking, Verification, and Optimization
- StitchCUDA: An Automated Multi-Agents End-to-End GPU Programing Framework with Rubric-based Agentic Reinforcement Learning
Related papers
- A Physics-Regulated Neural Framework for Learning 3D Grain Growth Dynamics
- FlexMoE: Learning One-for-All Multi-Budget Expert Pruning for Flexible MoE Deployment
- OptiMind: Improving Small Language Models for Optimization Modeling through Formulation Guidance and Multi-Turn Learning
- Staying in the Sweet Spot: Boost Reasoning Evolution via Responsive Rollout Difficulty Manipulation
- Examining the Impact of Data and Training Methods on Emergent and Subliminal Misalignment
- Rethinking Black-Box On-Policy Distillation: A Rubric-based Analysis