SWE-bench Bash Only — Official SWE-bench "Bash Only" agentless split, swe_bench_official source
| # | Model | Vendor | Best score | Runs | Last seen |
|---|---|---|---|---|---|
| 1 | Claude Opus 4.5 | Anthropic | 76.8 | 4 | 2026-08-10 |
| 2 | MiniMax M2.5 | MiniMax | 75.8 | 1 | 2026-05-01 |
| 3 | Gemini 3 Flash Preview | 75.8 | 2 | 2026-05-30 | |
| 4 | Claude Opus 4.6 | Anthropic | 75.6 | 1 | 2026-05-01 |
| 5 | Gemini 3 Pro | 74.2 | 2 | 2026-05-01 | |
| 6 | GPT-5.2 Codex | OpenAI | 72.8 | 1 | 2026-05-01 |
| 7 | GPT-5.2 | OpenAI | 72.8 | 4 | 2026-08-10 |
| 8 | GLM-5 | Z.ai | 72.8 | 1 | 2026-05-01 |
| 9 | Claude Sonnet 4.5 | Anthropic | 71.4 | 6 | 2026-08-10 |
| 10 | Kimi K2.5 | Moonshot | 70.8 | 1 | 2026-05-01 |
| 11 | Claude 4 Opus | Anthropic | 67.6 | 1 | 2026-05-01 |
| 12 | Claude Haiku 4.5 | Anthropic | 66.6 | 1 | 2026-05-01 |
| 13 | GPT-5.1 Codex | OpenAI | 66 | 2 | 2026-05-30 |
| 14 | GPT-5.1 | OpenAI | 66 | 4 | 2026-06-21 |
| 15 | GPT-5 | OpenAI | 65 | 3 | 2026-08-10 |
| 16 | Claude Sonnet 4 | Anthropic | 64.9 | 1 | 2026-05-01 |
| 17 | Kimi K2 (Reasoning) | Moonshot | 63.4 | 1 | 2026-05-01 |
| 18 | MiniMax M2 | MiniMax | 61 | 1 | 2026-05-01 |
| 19 | DeepSeek-V3.2 | DeepSeek | 60 | 1 | 2026-05-01 |
| 20 | GPT-5 mini | OpenAI | 59.8 | 2 | 2026-05-01 |
| 21 | o3 | OpenAI | 58.4 | 1 | 2026-05-01 |
| 22 | Qwen3 Coder 480B A35B Instruct | Alibaba | 55.4 | 2 | 2026-06-19 |
| 23 | GLM-4.6 | Z.ai | 55.4 | 1 | 2026-05-01 |
| 24 | GLM-4.5 | Z.ai | 54.2 | 1 | 2026-05-01 |
| 25 | Devstral 2512 | Mistral | 53.8 | 1 | 2026-05-01 |
| 26 | Gemini 2.5 Pro | 53.6 | 1 | 2026-05-01 | |
| 27 | Claude 3.7 Sonnet | Anthropic | 52.8 | 2 | 2026-06-21 |
| 28 | o4-mini | OpenAI | 45 | 1 | 2026-05-01 |
| 29 | Kimi K2 Instruct | Moonshot | 43.8 | 1 | 2026-05-01 |
| 30 | GPT-4.1 | OpenAI | 39.6 | 1 | 2026-05-01 |
| 31 | GPT-5 nano | OpenAI | 34.8 | 1 | 2026-05-01 |
| 32 | Gemini 2.5 Flash | 28.7 | 1 | 2026-05-01 | |
| 33 | GPT OSS 120B | OpenAI | 26 | 1 | 2026-05-01 |
| 34 | GPT-4.1 mini | OpenAI | 23.9 | 2 | 2026-06-19 |
| 35 | GPT-4o | OpenAI | 21.6 | 1 | 2026-05-01 |
| 36 | Llama 4 Maverick | Meta | 21 | 1 | 2026-05-01 |
| 37 | Gemini 2.0 Flash (Reasoning) | 13.5 | 1 | 2026-06-24 | |
| 38 | Gemini 2.0 Flash (Non-Reasoning) | 13.5 | 1 | 2026-05-01 | |
| 39 | Llama 4 Scout | Meta | 9.1 | 1 | 2026-05-01 |
| 40 | Qwen 2.5 Coder 32B Instruct | Alibaba | 9 | 1 | 2026-05-01 |
Best tracked score per model (default configuration; source-attributed and verification-tiered). Open a model for its full benchmark surface, provenance and pricing.