Which is better, Claude 3.5 Sonnet or Qwen2.5 Omni 7B?

Refresh Models —Deals —Regulatory —Sources —

Questions this page answers

Which is better, Claude 3.5 Sonnet or Qwen2.5 Omni 7B?: Across 13 shared benchmarks, Claude 3.5 Sonnet scores higher on 9 and Qwen2.5 Omni 7B on 3, with 1 level. The widest gap is GPQA Diamond, where Claude 3.5 Sonnet scores 67.2 against 30.8. Qwen2.5 Omni 7B is the cheaper of the two on tracked API pricing ($0.10 against $3.00 per million input tokens).

Claude 3.5 Sonnet vs Qwen2.5 Omni 7B

Across 13 shared benchmarks, Claude 3.5 Sonnet scores higher on 9 and Qwen2.5 Omni 7B on 3, with 1 level. The widest gap is GPQA Diamond, where Claude 3.5 Sonnet scores 67.2 against 30.8. Qwen2.5 Omni 7B is the cheaper of the two on tracked API pricing ($0.10 against $3.00 per million input tokens).

AnthropicvsAlibaba13 shared benchmarks9–3 head-to-head

Benchmark	Claude 3.5 Sonnet	Qwen2.5 Omni 7B
AI2D	94.7	83.2
chartqa	90.8	85.3
docvqa	95.2	95.2
GPQA Diamond	67.2	30.8
GSM8K	96.9	88.7
humaneval	93.7	78.7
MathVista	67.7	67.9
mmlu_redux	88.9	71
MMLU-Pro	78	47
MMMU	72	59.2
MMMU-Pro	54.7	36.6
MMStar	62.2	64
RealWorldQA	60.1	70.3

Best tracked score per model per benchmark (default configuration; source-attributed). ↓ marks lower-is-better metrics. Open either model for its full surface, provenance and pricing. Quoted rates are the price-setter row we currently track for each model — its direct or vendor-official listing where one exists (direct, alibaba-official), otherwise the lowest tracked offer.