SWE Atlas
“We evaluate a range of frontier and open-weight models on SWE Atlas and find that GPT-5.4 and Opus 4.7 achieve the strongest overall performance, while even the best open-weight models score poorly.”
The authors, in the abstract
Benchmarking Coding Agents Beyond Issue Resolution