SREGym: A Live Benchmark for AI SRE Agents with High-Fidelity Failure Scenarios
cs.AI
Submitted: 2026-05-08
Updated: 2026-11-15
Code: https://github.com/SREGym/SR
Terminology
Sources
- OpsEval: A Comprehensive IT Operations Benchmark Suite for Large Language Models
- OpsAgent: An Evolving Multi-agent System for Incident Management in Microservices
- StepFly: Agentic Troubleshooting Guide Automation for Incident Diagnosis
- Terminal-Bench: Benchmarking Agents on Hard, Realistic Tasks in Command Line Interfaces
- MicroRCA-Agent: Microservice Root Cause Analysis Method Based on Large Language Model Agents
- GALA: Can Graph-Augmented Large Language Model Agentic Workflows Elevate Root Cause Analysis?
- Cloud-OpsBench: A Reproducible Benchmark for Agentic Root Cause Analysis in Cloud Systems
- Adaptive Root Cause Localization for Microservice Systems with Multi-Agent Recursion-of-Thought
- MicroRemed: Benchmarking LLMs in Microservices Remediation
- Configuration Defects in Kubernetes
- Establishing Best Practices for Building Rigorous Agentic Benchmarks
Related papers
- Multi-Objective Structured Pruning of LLMs for Latency and Model Size Optimization
- MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
- Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
- Embeddings for Preferences, Not Semantics
- Can We Trust Item Response Theory for AI Evaluation?
- Diagnosing Task Insensitivity in Language Agents