Llama 3 (70B) is the stronger all-rounder.
Scores updated · 19 tests both models report · How we compare
Where each one wins
Tests won in each of the one area where both have results. Each piece is one test, so a longer bar means more evidence; grey means the two scored within a point of each other.
Reasoning rests on a single test.
The biggest differences
The tests each model wins by the widest margin, up to three each. Scores are out of 100.
Where Llama 3 (70B) pulls ahead
- Graduate-level biology, physics and chemistry questionsGPQA Diamond+10.6points ahead
Where Gemma (7B) pulls ahead
No clear win on a test scored out of 100.
Every test, side by side
All 19 tests both models report. The winning score is in its model's colour; marks a score checked independently.
Other results18 tests, not counted
Tests outside the eight areas. They are not counted above: several are summary scores built from other tests, or the same test under another name.
- Multi-MathematicsLlama 3 (70B) by 2839.167.1+28
- BBHLlama 3 (70B) by 25.955.181+25.9
- GSM8KLlama 3 (70B) by 24.655.980.5+24.6
- C-EvalLlama 3 (70B) by 21.643.665.2+21.6
- MBPPLlama 3 (70B) by 19.850.670.4+19.8
- MMLU-ProLlama 3 (70B) by 19.133.752.8+19.1
- MultiPL-ELlama 3 (70B) by 16.629.746.3+16.6
- MATHLlama 3 (70B) by 16.35066.3+16.3
- EvalPlusLlama 3 (70B) by 15.239.654.8+15.2
- naturalquestions_closedbookLlama 3 (70B) by 13.933.647.5+13.9
- OpenBookQALlama 3 (70B) by 12.680.893.4+12.6
- MMLULlama 3 (70B) by 12.457.169.5+12.4
- HumanEvalLlama 3 (70B) by 1137.248.2+11
- Theorem QALlama 3 (70B) by 10.821.532.3+10.8
- ARC-ChallengeLlama 3 (70B) by 7.761.168.8+7.7
- HellaSwagLlama 3 (70B) by 5.882.288+5.8
- NarrativeQALlama 3 (70B) by 4.675.279.8+4.6
- TruthfulQAtie44.845.6tie
Questions people ask
Which is better, Gemma (7B) or Llama 3 (70B)?
Llama 3 (70B) wins the one area where both have results: reasoning. Gemma (7B) wins none.
How do you compare the two?
We use the 19 benchmark tests both models have published scores on. The verdict counts the 1 tests in the eight capability areas, and a gap under one point (ten on rating-style scales) counts as a tie. The other 18 are listed but not counted, because several are summary scores or repeat a test. Each score is the one shown on the model's own page.