Claude Sonnet 4 vs Mistral Small 3 (Non-Reasoning)
Wins 4 of 5 areas
Coding · Reasoning · Long documents · Following instructions
Wins 1 of 5 areas
Facts
Claude Sonnet 4 is the stronger all-rounder.Mistral Small 3 (Non-Reasoning) is cheaper and better at facts.
Scores updated · 17 tests both models report · How we compare
Where each one wins
Tests won in each of the five areas where both have results. Each piece is one test, so a longer bar means more evidence; grey means the two scored within a point of each other.
- ReasoningHard problems that need careful thinking20Claude Sonnet 42 of 2 tests
- CodingWriting and fixing software10Claude Sonnet 41 of 1 test
- Long documentsFinding answers in very long texts10Claude Sonnet 41 of 1 test
- Following instructionsDoing exactly what it is asked10Claude Sonnet 41 of 1 test
- FactsGetting facts right instead of making them up01Mistral Small 3 (Non-Reasoning)1 of 1 test
Coding, facts, long documents and following instructions rest on a single test each.
What it costs
Prices per million tokens, roughly 750,000 words. The bars show the cost of a million tokens read plus a million written.
Mistral Small 3 (Non-Reasoning) costs 99% less for the same work.
The biggest differences
The tests each model wins by the widest margin, up to three each. Scores are out of 100.
Where Claude Sonnet 4 pulls ahead
- Graduate-level biology, physics and chemistry questionsGPQA Diamond+31.5points ahead
- Follows unfamiliar, precisely checkable instructionsIFBench+28.3points ahead
- Code for real scientific research problemsSciCode+16.4points ahead
Where Mistral Small 3 (Non-Reasoning) pulls ahead
No clear win on a test scored out of 100.
Every test, side by side
All 17 tests both models report. The winning score is in its model's colour; marks a score checked independently.
ReasoningClaude Sonnet 4
- GPQA DiamondClaude Sonnet 4 by 31.577.746.2+31.5
- Humanity's Last ExamClaude Sonnet 4 by 6.910.73.8+6.9
FactsMistral Small 3 (Non-Reasoning)
- Vectara HHEM hallucination ratelower is betterMistral Small 3 (Non-Reasoning) by 5.210.35.1+5.2
Following instructionsClaude Sonnet 4
- IFBenchClaude Sonnet 4 by 28.354.726.4+28.3
Other results11 tests, not counted
Tests outside the eight areas. They are not counted above: several are summary scores built from other tests, or the same test under another name.
- AIR-Bench 2024Claude Sonnet 4 by 55.588.332.8+55.5
- HarmBenchClaude Sonnet 4 by 52.89845.2+52.8
- τ²-Bench Telecom (AA run)Claude Sonnet 4 by 4564.619.6+45
- AA IntelligenceClaude Sonnet 4 by 12.218.96.7+12.2
- simple_safety_testsClaude Sonnet 4 by 6.710093.3+6.7
- vectara_factual_consistencyMistral Small 3 (Non-Reasoning) by 5.289.794.9+5.2
- vectara_avg_summary_lengthClaude Sonnet 4 by 47 rating points145.898.8+47 rating
- bbqClaude Sonnet 4 by 3.396.693.3+3.3
- anthropic_red_teamClaude Sonnet 4 by 2.899.296.4+2.8
- XSTestClaude Sonnet 4 by 1.396.695.3+1.3
- vectara_answer_ratetie98.697.9tie
Questions people ask
Which is better, Claude Sonnet 4 or Mistral Small 3 (Non-Reasoning)?
Claude Sonnet 4 wins four of the five areas where both have results: coding, reasoning, long documents and following instructions. Mistral Small 3 (Non-Reasoning) wins facts, and costs 99% less.
Which is better for coding?
Claude Sonnet 4. It wins the one coding test both models report.
Which is cheaper?
Claude Sonnet 4 costs $3.00 per million input tokens and $15.00 per million output tokens; Mistral Small 3 (Non-Reasoning) costs $0.05 and $0.08. That makes Mistral Small 3 (Non-Reasoning) about 99% cheaper for the same work.
How do you compare the two?
We use the 17 benchmark tests both models have published scores on. The verdict counts the 6 tests in the eight capability areas, and a gap under one point (ten on rating-style scales) counts as a tie. The other 11 are listed but not counted, because several are summary scores or repeat a test. Each score is the one shown on the model's own page.