Claude 3.5 Sonnet is the stronger all-rounder.
Scores updated · 24 tests both models report · How we compare
Where each one wins
Tests won in each of the two areas where both have results. Each piece is one test, so a longer bar means more evidence; grey means the two scored within a point of each other.
Reasoning and long documents rest on a single test each.
What it costs
Prices per million tokens, roughly 750,000 words. The bars show the cost of a million tokens read plus a million written.
The biggest differences
The tests each model wins by the widest margin, up to three each. Scores are out of 100.
Where Claude 3.5 Sonnet pulls ahead
- Graduate-level biology, physics and chemistry questionsGPQA Diamond+20.7points ahead
- Questions about very long textsLongBench v2+9.4points ahead
Where DeepSeek-V2 pulls ahead
No clear win on a test scored out of 100.
Every test, side by side
All 24 tests both models report. The winning score is in its model's colour; marks a score checked independently.
Long documentsClaude 3.5 Sonnet
- LongBench v2Claude 3.5 Sonnet by 9.44131.6+9.4
Other results22 tests, not counted
Tests outside the eight areas. They are not counted above: several are summary scores built from other tests, or the same test under another name.
- CodeforcesClaude 3.5 Sonnet by 699.5 rating points71717.5+699.5 rating
- HumanEvalClaude 3.5 Sonnet by 50.493.743.3+50.4
- IFEvalClaude 3.5 Sonnet by 28.886.557.7+28.8
- MGSMClaude 3.5 Sonnet by 2891.663.6+28
- MATHClaude 3.5 Sonnet by 27.771.143.4+27.7
- MMLU-ProClaude 3.5 Sonnet by 26.277.651.4+26.2
- Aider-Edit (Acc.)Claude 3.5 Sonnet by 23.984.260.3+23.9
- MATH-500 (EM)Claude 3.5 Sonnet by 2278.356.3+22
- LiveCodeBenchClaude 3.5 Sonnet by 21.232.811.6+21.2
- GSM8KClaude 3.5 Sonnet by 17.296.479.2+17.2
- BBHClaude 3.5 Sonnet by 14.393.178.8+14.3
- MMLUClaude 3.5 Sonnet by 13.188.775.6+13.1
- HumanEval-Mul (Pass@1)Claude 3.5 Sonnet by 12.481.769.3+12.4
- AIME 2024Claude 3.5 Sonnet by 11.4164.6+11.4
- MMLU-ReduxClaude 3.5 Sonnet by 1188.977.9+11
- DROP (F1)Claude 3.5 Sonnet by 8.488.880.4+8.4
- DROPClaude 3.5 Sonnet by 787.180.1+7
- Chinese SimpleQA (C-SimpleQA)Claude 3.5 Sonnet by 6.955.448.5+6.9
- FRAMES (Acc.)Claude 3.5 Sonnet by 5.672.566.9+5.6
- DROP (3-shot F1)Claude 3.5 Sonnet by 5.388.383+5.3
- C-EvalDeepSeek-V2 by 4.776.781.4+4.7
- CLUEWSCClaude 3.5 Sonnet by 3.485.482+3.4
Questions people ask
Which is better, Claude 3.5 Sonnet or DeepSeek-V2?
Claude 3.5 Sonnet wins all two areas where both have results: reasoning and long documents. DeepSeek-V2 wins none.
How do you compare the two?
We use the 24 benchmark tests both models have published scores on. The verdict counts the 2 tests in the eight capability areas, and a gap under one point (ten on rating-style scales) counts as a tie. The other 22 are listed but not counted, because several are summary scores or repeat a test. Each score is the one shown on the model's own page.