Claude 3.7 Sonnet vs Gemini 2.5 Flash (Sep) (Non-Reasoning)
Wins 2 of 7 areas
Coding · Facts
Wins 5 of 7 areas
Agents · Reasoning · Images and charts · Long documents · Following instructions
Gemini 2.5 Flash (Sep) (Non-Reasoning) is the stronger all-rounder.Claude 3.7 Sonnet is better at coding.
Scores updated · 17 tests both models report · How we compare
Where each one wins
Tests won in each of the seven areas where both have results. Each piece is one test, so a longer bar means more evidence; grey means the two scored within a point of each other.
- ReasoningHard problems that need careful thinking02Gemini 2.5 Flash (Sep) (Non-Reasoning)2 of 3 tests · 1 tie
- Images and chartsUnderstanding pictures, charts and video02Gemini 2.5 Flash (Sep) (Non-Reasoning)2 of 2 tests
- AgentsCarrying out multi-step tasks on its own01Gemini 2.5 Flash (Sep) (Non-Reasoning)1 of 1 test
- Long documentsFinding answers in very long texts01Gemini 2.5 Flash (Sep) (Non-Reasoning)1 of 1 test
- Following instructionsDoing exactly what it is asked01Gemini 2.5 Flash (Sep) (Non-Reasoning)1 of 1 test
- CodingWriting and fixing software10Claude 3.7 Sonnet1 of 2 tests · 1 tie
- FactsGetting facts right instead of making them up10Claude 3.7 Sonnet1 of 2 tests · 1 tie
Agents, long documents and following instructions rest on a single test each.
What it costs
Prices per million tokens, roughly 750,000 words. The bars show the cost of a million tokens read plus a million written.
The biggest differences
The tests each model wins by the widest margin, up to three each. Scores are out of 100.
Where Gemini 2.5 Flash (Sep) (Non-Reasoning) pulls ahead
- Reasons across sets of long documentsAA-LCR+19.3points ahead
- Harder college exam questions with imagesMMMU-Pro+13points ahead
- Very hard expert questions across many subjectsHumanity's Last Exam+4.1points ahead
Where Claude 3.7 Sonnet pulls ahead
- Avoids making up answers it doesn't knowAA-Omniscience · Non-hallucination+49.9points ahead
- Hard command-line tasks in a real terminalTerminal-Bench Hard+4.5points ahead
Every test, side by side
All 17 tests both models report. The winning score is in its model's colour; marks a score checked independently.
CodingClaude 3.7 Sonnet
- Terminal-Bench HardClaude 3.7 Sonnet by 4.521.216.7+4.5
- SciCodetie40.340.5tie
AgentsGemini 2.5 Flash (Sep) (Non-Reasoning)
- GDPValGemini 2.5 Flash (Sep) (Non-Reasoning) by 1.127.428.5+1.1
ReasoningGemini 2.5 Flash (Sep) (Non-Reasoning)
- Humanity's Last ExamGemini 2.5 Flash (Sep) (Non-Reasoning) by 4.19.713.8+4.1
- GPQA DiamondGemini 2.5 Flash (Sep) (Non-Reasoning) by 2.177.279.3+2.1
- CritPttie0.90.3tie
FactsClaude 3.7 Sonnet
- AA-Omniscience · Non-hallucinationClaude 3.7 Sonnet by 49.96010.1+49.9
- AA-Omniscience · Accuracytie2828.3tie
Images and chartsGemini 2.5 Flash (Sep) (Non-Reasoning)
- MMMU-ProGemini 2.5 Flash (Sep) (Non-Reasoning) by 1360.173.1+13
- LMArena · VisionGemini 2.5 Flash (Sep) (Non-Reasoning) by 85 rating points11691254+85 rating
Long documentsGemini 2.5 Flash (Sep) (Non-Reasoning)
- AA-LCRGemini 2.5 Flash (Sep) (Non-Reasoning) by 19.351.771+19.3
Following instructionsGemini 2.5 Flash (Sep) (Non-Reasoning)
- IFBenchGemini 2.5 Flash (Sep) (Non-Reasoning) by 448.352.3+4
Other results5 tests, not counted
Tests outside the eight areas. They are not counted above: several are summary scores built from other tests, or the same test under another name.
- AA-OmniscienceClaude 3.7 Sonnet by 35.4-0.7-36.1+35.4
- Artificial Analysis Coding IndexClaude 3.7 Sonnet by 11.836.424.6+11.8
- τ²-Bench Telecom (AA run)Claude 3.7 Sonnet by 9.154.745.6+9.1
- AA Agentic IndexClaude 3.7 Sonnet by 2.33734.7+2.3
- AA IntelligenceClaude 3.7 Sonnet by 2.217.715.5+2.2
Questions people ask
Which is better, Claude 3.7 Sonnet or Gemini 2.5 Flash (Sep) (Non-Reasoning)?
Gemini 2.5 Flash (Sep) (Non-Reasoning) wins five of the seven areas where both have results: agents, reasoning, images and charts, long documents and following instructions. Claude 3.7 Sonnet wins coding and facts.
Which is better for coding?
Claude 3.7 Sonnet. It wins 1 of the 2 coding tests both models report; Gemini 2.5 Flash (Sep) (Non-Reasoning) wins none, and 1 is a tie.
How do you compare the two?
We use the 17 benchmark tests both models have published scores on. The verdict counts the 12 tests in the eight capability areas, and a gap under one point (ten on rating-style scales) counts as a tie. The other 5 are listed but not counted, because several are summary scores or repeat a test. Each score is the one shown on the model's own page.