Which is better, o3 or o4-mini?

Across 62 shared benchmarks tracked on Vector Wire, o3 leads on 47 and o4-mini leads on 15 — o3 leads on more. Strengths differ by axis; the table below shows every head-to-head number with provenance.

SEEDRefresh Models —Deals —Regulatory —Sources —

o3 vs o4-mini

OpenAIvsOpenAI62 shared benchmarks47–15 head-to-head

Benchmark	o3	o4-mini
AA Agentic Index	36.1	36.1
AA Intelligence	31.1	26.1
AA-LCR	73.3	60
AA-Omniscience	-15.3	-35.7
aider_polyglot	81.3	72
AIME 2025	89.2	92.7
AIR-Bench 2024	84.5	78.5
anthropic_red_team	98.3	98.2
arc_agi_1	60.8	58.7
ARC-AGI-2	6.5	6.1
arena_vision	1217	1201
Artificial Analysis Coding Index	38.4	25.6
bbq	97.9	94
BBQ Accuracy on Ambiguous Questions	0.9	0.8
BBQ Accuracy on Unambiguous Questions	0.9	0.9
BBQ P(not stereotyping \| ambiguous question, not unknown)	0.3	0.3
browsecomp	49.7	51.5
critpt	1.1	0.6
Fortress	16	21.5
frontiermath_tier_4	2.1	6.3
gdpval	12.8	25.4
GPQA Diamond	83.3	81.4
harmbench	98.4	97
HLE	20.3	16.5
IFBench	71.4	68.7
imo_2025	16.7	14.3
LiveCodeBench	84.7	84.5
livecodebench_easy	99.1	98.8
livecodebench_hard	66	62.9
livecodebench_medium	89.8	92.2
MathVista	86.8	84.3
MMLU Language (0-shot) - Arabic	0.9	0.9
MMLU Language (0-shot) - Average	0.9	0.9
MMLU Language (0-shot) - Chinese (Simplified)	0.9	0.9
MMLU Language (0-shot) - French	0.9	0.9
MMLU Language (0-shot) - German	0.9	0.9
MMLU Language (0-shot) - Hindi	0.9	0.9
MMLU Language (0-shot) - Indonesian	0.9	0.9
MMLU Language (0-shot) - Italian	0.9	0.9
MMLU Language (0-shot) - Korean	0.9	0.9
MMMU	82.9	81.6
MMMU-Pro	76.4	69.2
multichallenge	60.4	44.9
MultiNRC	45.5	22.2
OmniScience Accuracy	38.6	24.8
OmniScience Non-Hallucination	12.9	19.5
PersonQA hallucination rate	0.3	0.5
PropensityBench	10.5	15.8
scicode	41	46.5
simple_safety_tests	99	100
simplebench	53.1	38.7
SimpleQA hallucination rate	0.5	0.8
simpleqa_verified	53	23.9
strongreject	97	96
swe_bench_bash	58.4	45
SWE-bench Verified	69.1	68.1
TAU-bench (airline)	52	49.2
TAU-bench (retail)	73.9	71.8
Terminal-Bench Hard	37.1	15.2
VisualToolBench	13.7	11.1
xstest	97.3	97.4
τ²-Bench Telecom (AA run)	80.7	55.6

Best tracked score per model per benchmark (default configuration; source-attributed). ↓ marks lower-is-better metrics. Open either model for its full surface, provenance and pricing.