Which is better, o1 or o4-mini?

Across 50 shared benchmarks tracked on Vector Wire, o1 leads on 26 and o4-mini leads on 24 — o1 leads on more. Strengths differ by axis; the table below shows every head-to-head number with provenance.

SEEDRefresh Models —Deals —Regulatory —Sources —

o1 vs o4-mini

OpenAIvsOpenAI50 shared benchmarks26–24 head-to-head

Benchmark	o1	o4-mini
AA Agentic Index	31.1	36.1
AA Intelligence	24	26.1
AA-LCR	63.3	60
AA-Omniscience	-10.6	-35.7
aider_polyglot	61.7	72
AIME 2025	81.7	92.7
AIR-Bench 2024	80	78.5
anthropic_red_team	98.3	98.2
arena_vision	1193	1201
Artificial Analysis Coding Index	39.7	25.6
bbq	97.3	94
BBQ Accuracy on Ambiguous Questions	1	0.8
BBQ Accuracy on Unambiguous Questions	0.9	0.9
BBQ P(not stereotyping \| ambiguous question, not unknown)	0.1	0.3
critpt	0.3	0.6
Fortress	19.4	21.5
gdpval	11.5	25.4
GPQA Diamond	75.7	81.4
GSM8K	97.1	0
harmbench	96.3	97
HLE	7.7	16.5
IFBench	70.3	68.7
MathVista	71.8	84.3
MMLU Language (0-shot) - Average	0.9	0.9
MMLU Language (0-shot) - Chinese (Simplified)	0.9	0.9
MMLU Language (0-shot) - French	0.9	0.9
MMLU Language (0-shot) - Hindi	0.9	0.9
MMLU Language (0-shot) - Indonesian	0.9	0.9
MMLU Language (0-shot) - Italian	0.9	0.9
MMLU Language (0-shot) - Japanese	0.9	0.9
MMLU Language (0-shot) - Korean	0.9	0.9
MMLU Language (0-shot) - Portuguese (Brazil)	0.9	0.9
MMLU Language (0-shot) - Swahili	0.9	0.8
MMLU Language (0-shot) - Yoruba	0.8	0.7
MMMU	77.6	81.6
OmniScience Accuracy	34.7	24.8
OmniScience Non-Hallucination	30.7	19.5
PersonQA hallucination rate	0.2	0.5
scicode	35.8	46.5
SEAL VISTA	45.3	51.8
simple_safety_tests	99	100
simplebench	41.7	38.7
SimpleQA hallucination rate	0.4	0.8
strongreject	97	96
SWE-bench Verified	48.9	68.1
TAU-bench (airline)	50	49.2
TAU-bench (retail)	70.8	71.8
Terminal-Bench Hard	12.9	15.2
xstest	97	97.4
τ²-Bench Telecom (AA run)	62.6	55.6

Best tracked score per model per benchmark (default configuration; source-attributed). ↓ marks lower-is-better metrics. Open either model for its full surface, provenance and pricing.