Which is better, GPT-4o or Qwen2 VL 72B Instruct?

Refresh Models —Deals —Regulatory —Sources —

Questions this page answers

Which is better, GPT-4o or Qwen2 VL 72B Instruct?: Across 24 shared benchmarks, GPT-4o scores higher on 6 and Qwen2 VL 72B Instruct on 18. The widest gap is MMMU-Pro, where GPT-4o scores 59.9 against 46.2.

GPT-4o vs Qwen2 VL 72B Instruct

Across 24 shared benchmarks, GPT-4o scores higher on 6 and Qwen2 VL 72B Instruct on 18. The widest gap is MMMU-Pro, where GPT-4o scores 59.9 against 46.2.

OpenAIvsAlibaba24 shared benchmarks6–18 head-to-head

Benchmark	GPT-4o	Qwen2 VL 72B Instruct
Avg.	27.8	30.9
chartqa	88.1	88.3
docvqa	92.8	96.5
DocVQA_test	92.8	96.5
EgoSchema	72.2	77.9
EgoSchema_test	72.2	77.9
HallBench_avg	55	58.1
mathvision	30.4	25.9
MathVista	63.8	70.5
MMBench-CN_test	82.1	86.6
MMBench-EN_test	83.4	86.5
MMBench-V1.1_test	82.2	85.9
MME_sum	2329	2483
MMMU	72.2	64.5
MMMU (val) (Pass@1)	69.1	64.5
MMMU-Pro	59.9	46.2
MMStar	63.9	68.3
MMT-Bench_test	65.5	71.7
OCRBench	806	877
RealWorldQA	75.4	77.8
REVERIE_valid-unseen SR	31.6	31
VCR_en easy	91.5	91.9
Video-MME	77.2	77.8
Video-MME (wo subs)	71.9	71.2

Best tracked score per model per benchmark (default configuration; source-attributed). ↓ marks lower-is-better metrics. Open either model for its full surface, provenance and pricing.