SREGym: Can AI agents resolve production issues? Real-world SRE problems including metastable failures, misconfigurations, and many more. Live system environments. From the University of Illinois at Urbana-Champaign. To submit, open an issue with the submission label at github.com/SREGym/SREGym.
Interactive terminal available on desktop
SREGym-Lite-1004 results
Top results on the curated 17-fault cohort.
1 | Radar + Claude Code* | Claude Opus 5.5 (medium) | 100.0 | 100.0 | 100.0 | 36.6 | 65.9 | 554K |
2 | Devin Oncall* | Claude Opus 5.5 (high) | 100.0 | 100.0 | 100.0 | 76.1 | 141.0 | 1.24M |
3 | CloudThinker* | GPT-6 Astra (low) | 100.0 | 100.0 | 100.0 | 126.2 | 342.0 | 736K |
4 | Devin Oncall* | GPT-6 Astra (max) | 100.0 | 100.0 | 100.0 | 190.8 | 388.4 | 1.11M |
5 | CloudThinker* | GPT-6 Astra (max) | 100.0 | 100.0 | 100.0 | 310.0 | 675.0 | 1.74M |
* Third-party submissions. Results verified by the SREGym team.
Diag. Diagnosis success rate · Mit. Mitigation success rate · E2E End-to-end (both diagnosis and mitigation correct) · TTD Time-to-diagnose (seconds) · TTM Time-to-mitigate (seconds) · Tokens Mean token usage per run