DeepSeek-R1 vs Llama 3 Instruct 70B
Wins 4 of 5 areas
Coding · Reasoning · Long documents · Following instructions
Wins 0 of 5 areas
—
DeepSeek-R1 is the stronger all-rounder.Llama 3 Instruct 70B is cheaper.
Scores updated · 19 tests both models report · How we compare
Where each one wins
Tests won in each of the five areas where both have results. Each piece is one test, so a longer bar means more evidence; grey means the two scored within a point of each other.
- ReasoningHard problems that need careful thinking20DeepSeek-R12 of 3 tests · 1 tie
- CodingWriting and fixing software20DeepSeek-R12 of 2 tests
- Long documentsFinding answers in very long texts10DeepSeek-R11 of 1 test
- Following instructionsDoing exactly what it is asked10DeepSeek-R11 of 1 test
- FactsGetting facts right instead of making them up11Even1 each
Long documents and following instructions rest on a single test each.
What it costs
Prices per million tokens, roughly 750,000 words. The bars show the cost of a million tokens read plus a million written.
Llama 3 Instruct 70B costs 43% less for the same work.
The biggest differences
The tests each model wins by the widest margin, up to three each. Scores are out of 100.
Where DeepSeek-R1 pulls ahead
- Graduate-level biology, physics and chemistry questionsGPQA Diamond+32.9points ahead
- Code for real scientific research problemsSciCode+19.4points ahead
- Answers hard knowledge questions correctlyAA-Omniscience · Accuracy+12.8points ahead
Where Llama 3 Instruct 70B pulls ahead
- Avoids making up answers it doesn't knowAA-Omniscience · Non-hallucination+2.8points ahead
Every test, side by side
All 19 tests both models report. The winning score is in its model's colour; marks a score checked independently.
CodingDeepSeek-R1
- SciCodeDeepSeek-R1 by 19.438.318.9+19.4
- Terminal-Bench HardDeepSeek-R1 by 5.36.10.8+5.3
ReasoningDeepSeek-R1
- GPQA DiamondDeepSeek-R1 by 32.970.837.9+32.9
- Humanity's Last ExamDeepSeek-R1 by 48.54.5+4
- CritPttie0.60tie
FactsEven
- AA-Omniscience · AccuracyDeepSeek-R1 by 12.830.517.7+12.8
- AA-Omniscience · Non-hallucinationLlama 3 Instruct 70B by 2.89.712.5+2.8
Following instructionsDeepSeek-R1
- IFBenchDeepSeek-R1 by 1.93937.1+1.9
Other results10 tests, not counted
Tests outside the eight areas. They are not counted above: several are summary scores built from other tests, or the same test under another name.
- AA-OmniscienceDeepSeek-R1 by 22.1-32.2-54.3+22.1
- Artificial Analysis Coding IndexDeepSeek-R1 by 17.824.66.8+17.8
- HarmBenchLlama 3 Instruct 70B by 16.147.964+16.1
- AIR-Bench 2024Llama 3 Instruct 70B by 11.752.964.6+11.7
- τ²-Bench Telecom (AA run)DeepSeek-R1 by 11.411.40+11.4
- AA IntelligenceDeepSeek-R1 by 5.911.45.5+5.9
- bbqDeepSeek-R1 by 5.696.691+5.6
- XSTestLlama 3 Instruct 70B by 2.494.496.8+2.4
- simple_safety_testsLlama 3 Instruct 70B by 19899+1
- anthropic_red_teamtie97.296.7tie
Questions people ask
Which is better, DeepSeek-R1 or Llama 3 Instruct 70B?
DeepSeek-R1 wins four of the five areas where both have results: coding, reasoning, long documents and following instructions. Llama 3 Instruct 70B wins none, but costs 43% less. They are level on facts.
Which is better for coding?
DeepSeek-R1. It wins 2 of the 2 coding tests both models report; Llama 3 Instruct 70B wins none.
Which is cheaper?
DeepSeek-R1 costs $2.00 per million input tokens and $4.00 per million output tokens; Llama 3 Instruct 70B costs $0.65 and $2.75. That makes Llama 3 Instruct 70B about 43% cheaper for the same work.
How do you compare the two?
We use the 19 benchmark tests both models have published scores on. The verdict counts the 9 tests in the eight capability areas, and a gap under one point (ten on rating-style scales) counts as a tie. The other 10 are listed but not counted, because several are summary scores or repeat a test. Each score is the one shown on the model's own page.