Claude 3.5 Haiku vs Mi:dm K 2.5 Pro
Wins 1 of 6 areas
Long documents
Wins 3 of 6 areas
Agents · Reasoning · Following instructions
Mi:dm K 2.5 Pro wins more areas, narrowly.Claude 3.5 Haiku is better at long documents.
Scores updated · 13 tests both models report · How we compare
Where each one wins
Tests won in each of the six areas where both have results. Each piece is one test, so a longer bar means more evidence; grey means the two scored within a point of each other.
- ReasoningHard problems that need careful thinking02Mi:dm K 2.5 Pro2 of 3 tests · 1 tie
- AgentsCarrying out multi-step tasks on its own01Mi:dm K 2.5 Pro1 of 1 test
- Following instructionsDoing exactly what it is asked01Mi:dm K 2.5 Pro1 of 1 test
- Long documentsFinding answers in very long texts10Claude 3.5 Haiku1 of 1 test
- CodingWriting and fixing software00Even0 each · 1 tie
- FactsGetting facts right instead of making them up11Even1 each
Coding, agents, long documents and following instructions rest on a single test each.
The biggest differences
The tests each model wins by the widest margin, up to three each. Scores are out of 100.
Where Mi:dm K 2.5 Pro pulls ahead
- Graduate-level biology, physics and chemistry questionsGPQA Diamond+29.3points ahead
- Follows unfamiliar, precisely checkable instructionsIFBench+6.5points ahead
- Answers hard knowledge questions correctlyAA-Omniscience · Accuracy+4.7points ahead
Where Claude 3.5 Haiku pulls ahead
- Avoids making up answers it doesn't knowAA-Omniscience · Non-hallucination+52.8points ahead
- Reasons across sets of long documentsAA-LCR+18.3points ahead
Every test, side by side
All 13 tests both models report. The winning score is in its model's colour; marks a score checked independently.
ReasoningMi:dm K 2.5 Pro
- GPQA DiamondMi:dm K 2.5 Pro by 29.340.870.1+29.3
- Humanity's Last ExamMi:dm K 2.5 Pro by 4.53.68.1+4.5
- CritPttie00tie
FactsEven
- AA-Omniscience · Non-hallucinationClaude 3.5 Haiku by 52.858.96.1+52.8
- AA-Omniscience · AccuracyMi:dm K 2.5 Pro by 4.713.217.9+4.7
Following instructionsMi:dm K 2.5 Pro
- IFBenchMi:dm K 2.5 Pro by 6.542.849.3+6.5
Other results4 tests, not counted
Tests outside the eight areas. They are not counted above: several are summary scores built from other tests, or the same test under another name.
- τ²-Bench Telecom (AA run)Mi:dm K 2.5 Pro by 61.924.686.5+61.9
- AA-OmniscienceClaude 3.5 Haiku by 36.7-22.5-59.2+36.7
- AA Agentic IndexMi:dm K 2.5 Pro by 34.91.936.8+34.9
- AA IntelligenceMi:dm K 2.5 Pro by 2.18.911+2.1
Questions people ask
Which is better, Claude 3.5 Haiku or Mi:dm K 2.5 Pro?
Mi:dm K 2.5 Pro wins three of the six areas where both have results: agents, reasoning and following instructions. Claude 3.5 Haiku wins long documents. They are level on coding and facts.
Which is better for coding?
Neither. All 1 coding tests both models report are ties.
How do you compare the two?
We use the 13 benchmark tests both models have published scores on. The verdict counts the 9 tests in the eight capability areas, and a gap under one point (ten on rating-style scales) counts as a tie. The other 4 are listed but not counted, because several are summary scores or repeat a test. Each score is the one shown on the model's own page.