Which is better, GPT-4o or Llama 3.1 Instruct 405B?

Across 56 shared benchmarks tracked on Vector Wire, GPT-4o leads on 42 and Llama 3.1 Instruct 405B leads on 12 — GPT-4o leads on more. Strengths differ by axis; the table below shows every head-to-head number with provenance.

SEEDRefresh Models —Deals —Regulatory —Sources —

GPT-4o vs Llama 3.1 Instruct 405B

OpenAIvsMeta56 shared benchmarks42–12 head-to-head

Benchmark	GPT-4o	Llama 3.1 Instruct 405B
AA Agentic Index	9.7	6.3
AA Intelligence	11.2	8.5
AA-LCR	55	24.7
AA-Omniscience	-10.5	-17.1
aider_edit_acc	72.9	63.9
aider_polyglot	30.7	5.8
AIME 2024	9.3	23.3
AIR-Bench 2024	62.4	58.6
anthropic_red_team	99.1	96.5
Arena Hard	92.4	69.3
Artificial Analysis Coding Index	24.2	14.5
bbq	95.1	94.5
C-Eval	76	72.5
Chinese SimpleQA (C-SimpleQA)	64.6	54.7
CLUEWSC	87.9	84.7
Codeforces	759	25.3
critpt	0	0
DROP	83.4	84.8
DROP (3-shot F1)	83.7	88.7
DROP (F1)	89.2	92.5
Fortress	47.2	20.6
FRAMES (Acc.)	80.5	70
gdpval	0	0
GPQA Diamond	70.1	51.5
GSM8K	95.6	96.8
harmbench	82.9	62.7
HLE	5.3	4.2
humaneval	90.6	89
HumanEval-Mul (Pass@1)	80.5	77.2
IFBench	36	39
ifeval	84.3	88.6
IFEval (avg)	84.1	86.4
LiveCodeBench	38.3	30.1
livecodebench_pass1cot	34.2	28.4
longbench_v2	48.1	36.1
MATH	85.3	82.7
math_500_em	74.6	73.8
MBPP+ (EvalPlus-augmented)	76.2	73
mgsm	90.5	91.6
mmlu	88.1	88.6
mmlu_redux	88	86.2
MMLU-Pro	74.7	73.4
mmmlu	81.4	73.8
narrativeqa	80.4	74.9
naturalquestions_closedbook	50.1	45.6
OmniScience Accuracy	23.7	23.2
OmniScience Non-Hallucination	62.1	49
OpenBookQA	96.8	94
scicode	33.4	29.9
simple_safety_tests	98.5	98.8
simplebench	17.8	23
simpleqa	39.4	23.2
SWE-bench Verified	38.8	24.5
Terminal-Bench Hard	8.3	6.8
xstest	97.3	95.9
τ²-Bench Telecom (AA run)	28.9	19

Best tracked score per model per benchmark (default configuration; source-attributed). ↓ marks lower-is-better metrics. Open either model for its full surface, provenance and pricing.