Which is better, Claude 3.5 Sonnet or Qwen2.5 Instruct 72B?

Across 57 shared benchmarks tracked on Vector Wire, Claude 3.5 Sonnet leads on 41 and Qwen2.5 Instruct 72B leads on 14 — Claude 3.5 Sonnet leads on more. Strengths differ by axis; the table below shows every head-to-head number with provenance.

SEEDRefresh Models —Deals —Regulatory —Sources —

Claude 3.5 Sonnet vs Qwen2.5 Instruct 72B

AnthropicvsAlibaba57 shared benchmarks41–14 head-to-head

Benchmark	Claude 3.5 Sonnet	Qwen2.5 Instruct 72B
AA Intelligence	10	10
aider_edit_acc	84.2	65.4
aider_polyglot	45.3	7.6
AIME 2024	16	23.3
AIR-Bench 2024	85.9	59
anthropic_red_team	99.8	99.6
Arena Hard	87.6	81.2
Artificial Analysis Coding Index	30.2	11.9
bbh	93.1	79.8
bbq	94.9	95.4
C-Eval	76.7	89.2
Chinese SimpleQA (C-SimpleQA)	56.8	52.2
CLUEWSC	85.4	91.4
Codeforces	717	24.8
DROP	87.1	76.7
DROP (3-shot F1)	88.3	76.7
DROP (F1)	88.8	85
Fortress	13	56.4
FRAMES (Acc.)	72.5	69.8
GPQA Diamond	67.2	49.1
GSM8K	96.9	95.8
harmbench	98.1	72.8
HLE	3.9	4.2
humaneval	93.7	86.6
HumanEval-Mul (Pass@1)	81.7	77.3
ifeval	90.1	87.2
IFEval (avg)	90.1	87.2
LiveCodeBench	32.8	55.5
livecodebench_pass1cot	36.3	31.1
LongBench v2 easy (w/ CoT)	55.2	47.9
LongBench v2 easy (w/o CoT)	46.9	42.7
LongBench v2 hard (w/ CoT)	41.5	40.8
LongBench v2 hard (w/o CoT)	37.3	41.8
LongBench v2 long (w/ CoT)	44.4	39.8
LongBench v2 long (w/o CoT)	37	44.4
LongBench v2 medium (w/ CoT)	41.9	40.9
LongBench v2 medium (w/o CoT)	38.6	38.1
LongBench v2 overall (w/ CoT)	46.7	43.5
LongBench v2 overall (w/o CoT)	41	42.1
LongBench v2 short (w/ CoT)	53.9	48.9
LongBench v2 short (w/o CoT)	46.1	45.6
longbench_v2	41	39.4
MATH	81.3	88.4
math_500_em	78.3	80
MBPP+ (EvalPlus-augmented)	75.1	77
mgsm	91.6	87.3
mmlu	88.3	86.1
mmlu_redux	88.9	86.8
MMLU-Pro	78	71.6
narrativeqa	74.6	74.5
naturalquestions_closedbook	50.2	35.9
OpenBookQA	97.2	96.2
scicode	36.6	26.7
simple_safety_tests	100	100
simpleqa	28.4	10.3
SWE-bench Verified	50.8	23.8
xstest	95.6	97.9

Best tracked score per model per benchmark (default configuration; source-attributed). ↓ marks lower-is-better metrics. Open either model for its full surface, provenance and pricing.