Which is better, DeepSeek-V3 or Llama 3.1 Instruct 405B?

Across 94 shared benchmarks tracked on Vector Wire, DeepSeek-V3 leads on 68 and Llama 3.1 Instruct 405B leads on 23 — DeepSeek-V3 leads on more. Strengths differ by axis; the table below shows every head-to-head number with provenance.

SEEDRefresh Models —Deals —Regulatory —Sources —

DeepSeek-V3 vs Llama 3.1 Instruct 405B

DeepSeekvsMeta94 shared benchmarks68–23 head-to-head

Benchmark	DeepSeek-V3	Llama 3.1 Instruct 405B
AA Agentic Index	1.6	6.3
AA Intelligence	15.4	8.5
AA-LCR	41.3	24.7
AA-Omniscience	-37.6	-17.1
AGIEval	79.6	60.6
aider_edit_acc	79.7	63.9
aider_polyglot	55.1	5.8
AIME 2024	59.4	23.3
AIR-Bench 2024	40.8	58.6
anthropic_red_team	97.1	96.5
arc_challenge	95.3	96.9
ARC-Challenge (Acc.)	95.3	95.3
ARC-Easy (Acc.)	98.9	98.4
Arena Hard	91.4	69.3
Artificial Analysis Coding Index	23	14.5
bbh	87.5	85.9
BBH (EM)	87.5	82.9
bbq	96.7	94.5
C-Eval	90.1	72.5
C3	78.6	79.7
C3 (Acc.)	78.6	79.7
CCPM	92	78.6
Chinese SimpleQA (C-SimpleQA)	68	54.7
CLUEWSC	90.9	84.7
CMath (EM)	90.7	77.3
cmmlu	88.8	73.7
CMMLU (Acc.)	88.8	73.7
CMRC	76.3	76
Codeforces	1134	25.3
critpt	0	0
CRUXEval-I (Acc.)	67.3	58.5
CRUXEval-I (input prediction)	67.3	58.5
CRUXEval-O (Acc.)	69.8	59.9
CRUXEval-O (output prediction)	69.8	59.9
DROP	91.6	84.8
DROP (3-shot F1)	91.6	88.7
DROP (F1)	91	92.5
FRAMES (Acc.)	73.3	70
gdpval	0	0
GPQA Diamond	68.4	51.5
GSM8K	96.7	96.8
GSM8K (EM)	89.3	83.5
harmbench	49.7	62.7
hellaswag	88.9	89.2
HellaSwag (Acc.)	88.9	89.2
HLE	5.2	4.2
humaneval	92.1	89
HumanEval-Mul (Pass@1)	82.6	77.2
IFBench	41	39
ifeval	87.3	88.6
IFEval (avg)	87.3	86.4
LiveCodeBench	49.6	30.1
livecodebench_pass1cot	40.5	28.4
LiveCodeBench-Base (Pass@1)	19.4	15.5
longbench_v2	48.7	36.1
MATH	91.2	82.7
MATH (EM)	61.6	49
math_500_em	94	73.8
MBPP	75.4	73.4
MBPP+ (EvalPlus-augmented)	78.8	73
mgsm	79.8	91.6
MGSM (EM)	79.8	69.9
mmlu	89.4	88.6
MMLU (Acc.)	87.1	84.4
mmlu_redux	90.5	86.2
MMLU-Pro	81.2	73.4
MMLU-Pro (Acc.)	64.4	52.8
MMLU-Redux (Acc.)	86.2	81.3
mmmlu	79.4	73.8
MMMLU-non-English (Acc.)	79.4	73.8
narrativeqa	79.6	74.9
NaturalQuestions	40	41.5
NaturalQuestions (EM)	40	41.5
naturalquestions_closedbook	46.7	45.6
OmniScience Accuracy	25.4	23.2
OmniScience Non-Hallucination	23.3	49
OpenBookQA	95.4	94
Pile-test (BPB)	0.5	0.5
PIQA	84.7	85.9
PIQA (Acc.)	84.7	85.9
RACE-High	51.3	56.8
RACE-Middle	67.1	74.2
scicode	35.8	29.9
simple_safety_tests	95.3	98.8
simplebench	27.2	23
simpleqa	27.7	23.2
SWE-bench Verified	42	24.5
Terminal-Bench Hard	15.2	6.8
The Pile (Test, BPB)	0.5	0.5
TriviaQA (EM)	82.9	82.7
winogrande	84.9	86.7
WinoGrande (Acc.)	84.9	85.2
xstest	97.1	95.9
τ²-Bench Telecom (AA run)	47.1	19

Best tracked score per model per benchmark (default configuration; source-attributed). ↓ marks lower-is-better metrics. Open either model for its full surface, provenance and pricing.