Llama 3 Instruct 8B is the stronger all-rounder.
Scores updated · 26 tests both models report · How we compare
Where each one wins
Tests won in each of the one area where both have results. Each piece is one test, so a longer bar means more evidence; grey means the two scored within a point of each other.
Reasoning rests on a single test.
What it costs
Prices per million tokens, roughly 750,000 words. The bars show the cost of a million tokens read plus a million written.
The biggest differences
The tests each model wins by the widest margin, up to three each. Scores are out of 100.
Where Llama 3 Instruct 8B pulls ahead
- Graduate-level biology, physics and chemistry questionsGPQA Diamond+4.9points ahead
Where Mistral 7B pulls ahead
No clear win on a test scored out of 100.
Every test, side by side
All 26 tests both models report. The winning score is in its model's colour; marks a score checked independently.
ReasoningLlama 3 Instruct 8B
- GPQA DiamondLlama 3 Instruct 8B by 4.929.624.7+4.9
Other results25 tests, not counted
Tests outside the eight areas. They are not counted above: several are summary scores built from other tests, or the same test under another name.
- HumanEvalLlama 3 Instruct 8B by 31.160.429.3+31.1
- MATHLlama 3 Instruct 8B by 2639.113.1+26
- ARC-ChallengeLlama 3 Instruct 8B by 22.882.860+22.8
- TruthfulQALlama 3 Instruct 8B by 2163.242.2+21
- MBPPLlama 3 Instruct 8B by 16.667.751.1+16.6
- WinoGrandeMistral 7B by 13.46578.4+13.4
- HellaSwagMistral 7B by 12.171.183.2+12.1
- MedQALlama 3 Instruct 8B by 10.560.550+10.5
- ANLILlama 3 Instruct 8B by 10.257.347.1+10.2
- Multi-MathematicsLlama 3 Instruct 8B by 1036.326.3+10
- BoolQLlama 3 Instruct 8B by 8.780.972.2+8.7
- TriviaQAMistral 7B by 7.567.775.2+7.5
- AGI EVALLlama 3 Instruct 8B by 6.94235.1+6.9
- MultiPL-EMistral 7B by 6.822.629.4+6.8
- MMLU-ProLlama 3 Instruct 8B by 4.535.430.9+4.5
- MMLUMistral 7B by 460.164.2+4
- EvalPlusLlama 3 Instruct 8B by 3.940.336.4+3.9
- Popular aggregated benchmarkLlama 3 Instruct 8B by 3.459.956.5+3.4
- OpenBookQAMistral 7B by 3.276.679.8+3.2
- Theorem QALlama 3 Instruct 8B by 2.922.119.2+2.9
- GSM8KMistral 7B by 2.349.952.2+2.3
- C-EvalLlama 3 Instruct 8B by 2.149.547.4+2.1
- PIQAMistral 7B by 275.777.7+2
- BBHLlama 3 Instruct 8B by 1.657.756.1+1.6
- Social IQAtie73.974.6tie
Questions people ask
Which is better, Llama 3 Instruct 8B or Mistral 7B?
Llama 3 Instruct 8B wins the one area where both have results: reasoning. Mistral 7B wins none.
How do you compare the two?
We use the 26 benchmark tests both models have published scores on. The verdict counts the 1 tests in the eight capability areas, and a gap under one point (ten on rating-style scales) counts as a tie. The other 25 are listed but not counted, because several are summary scores or repeat a test. Each score is the one shown on the model's own page.