Gemini 1.5 Flash (001) vs GPT-4o mini
Wins 1 of 3 areas
Images and charts
Wins 2 of 3 areas
Coding · Reasoning
GPT-4o mini is the stronger all-rounder.Gemini 1.5 Flash (001) is better at images and charts.
Scores updated · 16 tests both models report · How we compare
Where each one wins
Tests won in each of the three areas where both have results. Each piece is one test, so a longer bar means more evidence; grey means the two scored within a point of each other.
Coding and images and charts rest on a single test each.
What it costs
Prices per million tokens, roughly 750,000 words. The bars show the cost of a million tokens read plus a million written.
The biggest differences
The tests each model wins by the widest margin, up to three each. Scores are out of 100.
Where GPT-4o mini pulls ahead
- Graduate-level biology, physics and chemistry questionsGPQA Diamond+10.2points ahead
- Code for real scientific research problemsSciCode+4.8points ahead
Where Gemini 1.5 Flash (001) pulls ahead
- Harder college exam questions with imagesMMMU-Pro+6.9points ahead
Every test, side by side
All 16 tests both models report. The winning score is in its model's colour; marks a score checked independently.
ReasoningGPT-4o mini
- GPQA DiamondGPT-4o mini by 10.232.442.6+10.2
- Humanity's Last Examtie4.44.2tie
Images and chartsGemini 1.5 Flash (001)
- MMMU-ProGemini 1.5 Flash (001) by 6.948.441.5+6.9
Other results12 tests, not counted
Tests outside the eight areas. They are not counted above: several are summary scores built from other tests, or the same test under another name.
- GSM8KGPT-4o mini by 51.532.884.3+51.5
- AIR-Bench 2024Gemini 1.5 Flash (001) by 10.867.156.3+10.8
- MATHGemini 1.5 Flash (001) by 10.690.880.2+10.6
- naturalquestions_closedbookGPT-4o mini by 6.332.338.5+6.3
- HarmBenchGPT-4o mini by 4.98084.9+4.9
- XSTestGPT-4o mini by 3.992.196+3.9
- NarrativeQAGPT-4o mini by 2.274.676.8+2.2
- anthropic_red_teamGemini 1.5 Flash (001) by 1.699.998.3+1.6
- MMLUGemini 1.5 Flash (001) by 1.167.966.8+1.1
- AA Intelligencetie5.96.7tie
- simple_safety_teststie9797.8tie
- OpenBookQAtie91.492tie
Questions people ask
Which is better, Gemini 1.5 Flash (001) or GPT-4o mini?
GPT-4o mini wins two of the three areas where both have results: coding and reasoning. Gemini 1.5 Flash (001) wins images and charts.
Which is better for coding?
GPT-4o mini. It wins the one coding test both models report.
How do you compare the two?
We use the 16 benchmark tests both models have published scores on. The verdict counts the 4 tests in the eight capability areas, and a gap under one point (ten on rating-style scales) counts as a tie. The other 12 are listed but not counted, because several are summary scores or repeat a test. Each score is the one shown on the model's own page.