Infrathon: Benchmarking Language Agents for Real-World, Long-Horizon AI Infra Optimization
cs.AI, cs.CL
Submitted: 2026-11-09
Updated: 2026-11-16
Code: https://github.com/InfrathonX/Infrathon
Project page: https://infrathonx.github.io
Terminology
Sources
- FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness
- Ansor: Generating High-Performance Tensor Programs for Deep Learning
- Daydream: Accurately Estimating the Efficacy of Optimizations for DNN Training
- KernelBench: Can LLMs Write Efficient GPU Kernels?
- InferenceBench: A Benchmark for Open-Ended LLM Inference Optimization by AI Agents
- EdgeBench: Unveiling Scaling Laws of Learning from Real-World Environments
- Liger Kernel: Efficient Triton Kernels for LLM Training
- FlashAttention-3: Fast and Accurate Attention with Asynchrony and Low-precision
- SageAttention: Accurate 8-Bit Attention for Plug-and-play Inference Acceleration
- Flash-KMeans: Fast and Memory-Efficient Exact K-Means
- Quant VideoGen: Auto-Regressive Long Video Generation via 2-Bit KV-Cache Quantization
- KernelBenchX: A Comprehensive Benchmark for Evaluating LLM-Generated GPU Kernels
- FlashInfer-Bench: Building the Virtuous Cycle for AI-driven LLM Systems
- RE-Bench: Evaluating frontier AI R&D capabilities of language model agents against human experts
- MLE-bench: Evaluating Machine Learning Agents on Machine Learning Engineering
- CORE-Bench: Fostering the Credibility of Published Research Through a Computational Reproducibility Agent Benchmark
- PaperBench: Evaluating AI's Ability to Replicate AI Research
- Continual Learning Bench: Evaluating Frontier AI Systems in Real-World Stateful Environments
- Long-Horizon-Terminal-Bench: Testing the Limits of Agents on Long-Horizon Terminal Tasks with Dense Reward-Based Grading
- Kevin: Multi-Turn RL for Generating CUDA Kernels
Related papers
- Multi-Objective Structured Pruning of LLMs for Latency and Model Size Optimization
- MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
- Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
- Embeddings for Preferences, Not Semantics
- Can We Trust Item Response Theory for AI Evaluation?
- Diagnosing Task Insensitivity in Language Agents