Which is better, Llama 3.1 Instruct 405B or Qwen2.5 Instruct 72B?

Across 94 shared benchmarks tracked on Vector Wire, Llama 3.1 Instruct 405B leads on 46 and Qwen2.5 Instruct 72B leads on 42 — Llama 3.1 Instruct 405B leads on more. Strengths differ by axis; the table below shows every head-to-head number with provenance.

SEEDRefresh Models —Deals —Regulatory —Sources —

Llama 3.1 Instruct 405B vs Qwen2.5 Instruct 72B

MetavsAlibaba94 shared benchmarks46–42 head-to-head

Benchmark	Llama 3.1 Instruct 405B	Qwen2.5 Instruct 72B
AA Intelligence	8.5	10
AA-LCR	24.7	20.3
AA-Omniscience	-17.1	-52.2
AGIEval	60.6	75.8
aider_edit_acc	63.9	65.4
aider_polyglot	5.8	7.6
AIME 2024	23.3	23.3
AIR-Bench 2024	58.6	59
AlignBench	6	81.6
anthropic_red_team	96.5	99.6
arc_challenge	96.9	94.5
ARC-Challenge (Acc.)	95.3	94.5
ARC-Easy (Acc.)	98.4	98.4
Arena Hard	69.3	81.2
Artificial Analysis Coding Index	14.5	11.9
bbh	85.9	79.8
BBH (EM)	82.9	79.8
bbq	94.5	95.4
C-Eval	72.5	89.2
C3	79.7	76.7
C3 (Acc.)	79.7	76.7
CCPM	78.6	88.5
Chinese SimpleQA (C-SimpleQA)	54.7	52.2
CLUEWSC	84.7	91.4
CMath (EM)	77.3	84.5
cmmlu	73.7	89.5
CMMLU (Acc.)	73.7	89.5
CMRC	76	75.8
Codeforces	25.3	24.8
critpt	0	0
CRUXEval-I (Acc.)	58.5	59.1
CRUXEval-I (input prediction)	58.5	59.1
CRUXEval-O (Acc.)	59.9	59.9
CRUXEval-O (output prediction)	59.9	59.9
DROP	84.8	76.7
DROP (3-shot F1)	88.7	76.7
DROP (F1)	92.5	85
Fortress	20.6	56.4
FRAMES (Acc.)	70	69.8
GPQA Diamond	51.5	49.1
GSM8K	96.8	95.8
GSM8K (EM)	83.5	88.3
harmbench	62.7	72.8
hellaswag	89.2	84.8
HellaSwag (Acc.)	89.2	84.8
HLE	4.2	4.2
humaneval	89	86.6
HumanEval-Mul (Pass@1)	77.2	77.3
IFBench	39	36.9
ifeval	88.6	87.2
IFEval (avg)	86.4	87.2
LiveCodeBench	30.1	55.5
livecodebench_pass1cot	28.4	31.1
LiveCodeBench-Base (Pass@1)	15.5	12.9
longbench_v2	36.1	39.4
MATH	82.7	88.4
MATH (EM)	49	54.4
math_500_em	73.8	80
MBPP	73.4	72.6
MBPP+ (EvalPlus-augmented)	73	77
mgsm	91.6	87.3
MGSM (EM)	69.9	76.2
mmlu	88.6	86.1
MMLU (Acc.)	84.4	85
mmlu_redux	86.2	86.8
MMLU-Pro	73.4	71.6
MMLU-Pro (Acc.)	52.8	58.3
MMLU-Redux (Acc.)	81.3	83.2
mmmlu	73.8	74.8
MMMLU-non-English (Acc.)	73.8	74.8
mt_bench	9.1	0.9
narrativeqa	74.9	74.5
NaturalQuestions	41.5	33.2
NaturalQuestions (EM)	41.5	33.2
naturalquestions_closedbook	45.6	35.9
OmniScience Accuracy	23.2	17.6
OmniScience Non-Hallucination	49	15.3
OpenBookQA	94	96.2
Pile-test (BPB)	0.5	0.6
PIQA	85.9	82.6
PIQA (Acc.)	85.9	82.6
RACE-High	56.8	50.3
RACE-Middle	74.2	68.1
scicode	29.9	26.7
simple_safety_tests	98.8	100
simpleqa	23.2	10.3
SWE-bench Verified	24.5	23.8
Terminal-Bench Hard	6.8	4.5
The Pile (Test, BPB)	0.5	0.6
TriviaQA (EM)	82.7	71.9
winogrande	86.7	82.3
WinoGrande (Acc.)	85.2	82.3
xstest	95.9	97.9
τ²-Bench Telecom (AA run)	19	34.5

Best tracked score per model per benchmark (default configuration; source-attributed). ↓ marks lower-is-better metrics. Open either model for its full surface, provenance and pricing.