Claude Sonnet 4 vs Llama 3 Instruct 70B
Wins 5 of 5 areas
Coding · Reasoning · Facts · Long documents · Following instructions
Wins 0 of 5 areas
—
Claude Sonnet 4 is the stronger all-rounder.Llama 3 Instruct 70B is cheaper.
Scores updated · 20 tests both models report · How we compare
Where each one wins
Tests won in each of the five areas where both have results. Each piece is one test, so a longer bar means more evidence; grey means the two scored within a point of each other.
- ReasoningHard problems that need careful thinking20Claude Sonnet 42 of 3 tests · 1 tie
- CodingWriting and fixing software20Claude Sonnet 42 of 2 tests
- FactsGetting facts right instead of making them up20Claude Sonnet 42 of 2 tests
- Long documentsFinding answers in very long texts10Claude Sonnet 41 of 1 test
- Following instructionsDoing exactly what it is asked10Claude Sonnet 41 of 1 test
Long documents and following instructions rest on a single test each.
What it costs
Prices per million tokens, roughly 750,000 words. The bars show the cost of a million tokens read plus a million written.
Llama 3 Instruct 70B costs 81% less for the same work.
The biggest differences
The tests each model wins by the widest margin, up to three each. Scores are out of 100.
Where Claude Sonnet 4 pulls ahead
- Avoids making up answers it doesn't knowAA-Omniscience · Non-hallucination+58.4points ahead
- Graduate-level biology, physics and chemistry questionsGPQA Diamond+39.8points ahead
- Hard command-line tasks in a real terminalTerminal-Bench Hard+30.3points ahead
Where Llama 3 Instruct 70B pulls ahead
No clear win on a test scored out of 100.
Every test, side by side
All 20 tests both models report. The winning score is in its model's colour; marks a score checked independently.
CodingClaude Sonnet 4
- Terminal-Bench HardClaude Sonnet 4 by 30.331.10.8+30.3
- SciCodeClaude Sonnet 4 by 21.14018.9+21.1
ReasoningClaude Sonnet 4
- GPQA DiamondClaude Sonnet 4 by 39.877.737.9+39.8
- Humanity's Last ExamClaude Sonnet 4 by 6.210.74.5+6.2
- CritPttie0.30tie
FactsClaude Sonnet 4
- AA-Omniscience · Non-hallucinationClaude Sonnet 4 by 58.470.912.5+58.4
- AA-Omniscience · AccuracyClaude Sonnet 4 by 522.717.7+5
Following instructionsClaude Sonnet 4
- IFBenchClaude Sonnet 4 by 17.654.737.1+17.6
Other results11 tests, not counted
Tests outside the eight areas. They are not counted above: several are summary scores built from other tests, or the same test under another name.
- τ²-Bench Telecom (AA run)Claude Sonnet 4 by 64.664.60+64.6
- AA-OmniscienceClaude Sonnet 4 by 54.50.2-54.3+54.5
- HarmBenchClaude Sonnet 4 by 349864+34
- Artificial Analysis Coding IndexClaude Sonnet 4 by 30.837.66.8+30.8
- AIR-Bench 2024Claude Sonnet 4 by 23.788.364.6+23.7
- AA Agentic IndexClaude Sonnet 4 by 17.617.60+17.6
- AA IntelligenceClaude Sonnet 4 by 13.418.95.5+13.4
- bbqClaude Sonnet 4 by 5.696.691+5.6
- anthropic_red_teamClaude Sonnet 4 by 2.599.296.7+2.5
- simple_safety_testsClaude Sonnet 4 by 110099+1
- XSTesttie96.696.8tie
Questions people ask
Which is better, Claude Sonnet 4 or Llama 3 Instruct 70B?
Claude Sonnet 4 wins all five areas where both have results: coding, reasoning, facts, long documents and following instructions. Llama 3 Instruct 70B wins none, but costs 81% less.
Which is better for coding?
Claude Sonnet 4. It wins 2 of the 2 coding tests both models report; Llama 3 Instruct 70B wins none.
Which is cheaper?
Claude Sonnet 4 costs $3.00 per million input tokens and $15.00 per million output tokens; Llama 3 Instruct 70B costs $0.65 and $2.75. That makes Llama 3 Instruct 70B about 81% cheaper for the same work.
How do you compare the two?
We use the 20 benchmark tests both models have published scores on. The verdict counts the 9 tests in the eight capability areas, and a gap under one point (ten on rating-style scales) counts as a tie. The other 11 are listed but not counted, because several are summary scores or repeat a test. Each score is the one shown on the model's own page.