Which is better, Gemini 1.5 Pro or Qwen2 VL 72B Instruct?

Refresh Models —Deals —Regulatory —Sources —

Questions this page answers

Which is better, Gemini 1.5 Pro or Qwen2 VL 72B Instruct?: Across 11 shared benchmarks, Gemini 1.5 Pro scores higher on 8 and Qwen2 VL 72B Instruct on 3. The widest gap is MMMU (val) (Pass@1), where Qwen2 VL 72B Instruct scores 64.5 against 54.1.

Gemini 1.5 Pro vs Qwen2 VL 72B Instruct

Across 11 shared benchmarks, Gemini 1.5 Pro scores higher on 8 and Qwen2 VL 72B Instruct on 3. The widest gap is MMMU (val) (Pass@1), where Qwen2 VL 72B Instruct scores 64.5 against 54.1.

GooglevsAlibaba11 shared benchmarks8–3 head-to-head

Benchmark	Gemini 1.5 Pro	Qwen2 VL 72B Instruct
chartqa	88.7	88.3
docvqa	91.5	96.5
MathVista	70.6	70.5
MMMU	68.4	64.5
MMMU (val) (Pass@1)	54.1	64.5
MMMU-Pro	55	46.2
OCRBench	800	877
ocrbench_v2	51.6	46.1
Video-MME	78.6	77.8
Video-MME (w subs)	81.3	77.8
Video-MME (wo subs)	75	71.2

Best tracked score per model per benchmark (default configuration; source-attributed). ↓ marks lower-is-better metrics. Open either model for its full surface, provenance and pricing.