DeepSeek-V3 wins more areas, narrowly.
Scores updated · 32 tests both models report · How we compare
Where each one wins
Tests won in each of the two areas where both have results. Each piece is one test, so a longer bar means more evidence; grey means the two scored within a point of each other.
What it costs
Prices per million tokens, roughly 750,000 words. The bars show the cost of a million tokens read plus a million written.
The biggest differences
The tests each model wins by the widest margin, up to three each. Scores are out of 100.
Where DeepSeek-V3 pulls ahead
- Code for real scientific research problemsSciCode+9.5points ahead
- Fixes real GitHub issues in Python projectsSWE-bench Verified+7.8points ahead
- Graduate-level biology, physics and chemistry questionsGPQA Diamond+6.6points ahead
Where Gemini 1.5 Pro pulls ahead
- Common-sense trick questionsSimpleBench+8.2points ahead
Every test, side by side
All 32 tests both models report. The winning score is in its model's colour; marks a score checked independently.
CodingDeepSeek-V3
- SciCodeDeepSeek-V3 by 9.53929.5+9.5
- SWE-bench VerifiedDeepSeek-V3 by 7.84234.2+7.8
ReasoningEven
- SimpleBenchGemini 1.5 Pro by 8.218.927.1+8.2
- GPQA DiamondDeepSeek-V3 by 6.665.558.9+6.6
- Humanity's Last Examtie4.74.6tie
Other results27 tests, not counted
Tests outside the eight areas. They are not counted above: several are summary scores built from other tests, or the same test under another name.
- HarmBenchGemini 1.5 Pro by 30.249.779.9+30.2
- AIR-Bench 2024Gemini 1.5 Pro by 26.540.867.3+26.5
- HumanEvalGemini 1.5 Pro by 18.965.284.1+18.9
- LiveCodeBenchDeepSeek-V3 by 18.749.230.5+18.7
- DROPDeepSeek-V3 by 16.791.674.9+16.7
- MMLUDeepSeek-V3 by 988.579.5+9
- Chinese SimpleQA (C-SimpleQA)DeepSeek-V3 by 8.66859.4+8.6
- MATHDeepSeek-V3 by 7.790.282.5+7.7
- MGSMGemini 1.5 Pro by 7.779.887.5+7.7
- Arena HardDeepSeek-V3 by 6.191.485.3+6.1
- HellaSwagGemini 1.5 Pro by 4.488.993.3+4.4
- NarrativeQADeepSeek-V3 by 479.675.6+4
- MBPP+ (EvalPlus-augmented)DeepSeek-V3 by 3.478.875.4+3.4
- IFEvalGemini 1.5 Pro by 3.386.189.4+3.3
- GSM8KDeepSeek-V3 by 3.29490.8+3.2
- anthropic_red_teamGemini 1.5 Pro by 2.897.199.9+2.8
- simple_safety_testsGemini 1.5 Pro by 2.295.397.5+2.2
- IFEval (avg)Gemini 1.5 Pro by 2.187.389.4+2.1
- AA IntelligenceDeepSeek-V3 by 1.89.77.9+1.8
- BBHGemini 1.5 Pro by 1.787.589.2+1.7
- XSTestGemini 1.5 Pro by 1.797.198.8+1.7
- SimpleQADeepSeek-V3 by 1.524.923.4+1.5
- naturalquestions_closedbookDeepSeek-V3 by 1.146.745.5+1.1
- Artificial Analysis Coding Indextie2323.6tie
- DROP (F1)tie8989.2tie
- OpenBookQAtie95.495.2tie
- MMLU-Protie75.975.8tie
Questions people ask
Which is better, DeepSeek-V3 or Gemini 1.5 Pro?
DeepSeek-V3 wins one of the two areas where both have results: coding. Gemini 1.5 Pro wins none. They are level on reasoning.
Which is better for coding?
DeepSeek-V3. It wins 2 of the 2 coding tests both models report; Gemini 1.5 Pro wins none.
How do you compare the two?
We use the 32 benchmark tests both models have published scores on. The verdict counts the 5 tests in the eight capability areas, and a gap under one point (ten on rating-style scales) counts as a tie. The other 27 are listed but not counted, because several are summary scores or repeat a test. Each score is the one shown on the model's own page.