Llama 3.1 Nemotron Instruct 70B vs Phi 4 Mini Instruct
Wins 4 of 6 areas
Coding · Reasoning · Facts · Following instructions
Wins 1 of 6 areas
Long documents
Llama 3.1 Nemotron Instruct 70B is the stronger all-rounder.Phi 4 Mini Instruct is better at long documents.
Scores updated · 20 tests both models report · How we compare
Where each one wins
Tests won in each of the six areas where both have results. Each piece is one test, so a longer bar means more evidence; grey means the two scored within a point of each other.
- CodingWriting and fixing software20Llama 3.1 Nemotron Instruct 70B2 of 2 tests
- FactsGetting facts right instead of making them up20Llama 3.1 Nemotron Instruct 70B2 of 2 tests
- ReasoningHard problems that need careful thinking10Llama 3.1 Nemotron Instruct 70B1 of 3 tests · 2 ties
- Following instructionsDoing exactly what it is asked10Llama 3.1 Nemotron Instruct 70B1 of 1 test
- Long documentsFinding answers in very long texts01Phi 4 Mini Instruct1 of 1 test
- AgentsCarrying out multi-step tasks on its own00Even0 each · 1 tie
Agents, long documents and following instructions rest on a single test each.
The biggest differences
The tests each model wins by the widest margin, up to three each. Scores are out of 100.
Where Llama 3.1 Nemotron Instruct 70B pulls ahead
- Graduate-level biology, physics and chemistry questionsGPQA Diamond+13.4points ahead
- Code for real scientific research problemsSciCode+12.5points ahead
- Follows unfamiliar, precisely checkable instructionsIFBench+9.6points ahead
Where Phi 4 Mini Instruct pulls ahead
- Reasons across sets of long documentsAA-LCR+7points ahead
Every test, side by side
All 20 tests both models report. The winning score is in its model's colour; marks a score checked independently.
CodingLlama 3.1 Nemotron Instruct 70B
- SciCodeLlama 3.1 Nemotron Instruct 70B by 12.523.310.8+12.5
- Terminal-Bench HardLlama 3.1 Nemotron Instruct 70B by 4.54.50+4.5
ReasoningLlama 3.1 Nemotron Instruct 70B
- GPQA DiamondLlama 3.1 Nemotron Instruct 70B by 13.446.533.1+13.4
- Humanity's Last Examtie4.24.5tie
- CritPttie00tie
FactsLlama 3.1 Nemotron Instruct 70B
- AA-Omniscience · AccuracyLlama 3.1 Nemotron Instruct 70B by 8.317.89.5+8.3
- AA-Omniscience · Non-hallucinationLlama 3.1 Nemotron Instruct 70B by 6.728.722+6.7
Following instructionsLlama 3.1 Nemotron Instruct 70B
- IFBenchLlama 3.1 Nemotron Instruct 70B by 9.630.721.1+9.6
Other results10 tests, not counted
Tests outside the eight areas. They are not counted above: several are summary scores built from other tests, or the same test under another name.
- Arena HardLlama 3.1 Nemotron Instruct 70B by 52.28532.8+52.2
- AA-OmniscienceLlama 3.1 Nemotron Instruct 70B by 20.3-40.8-61.1+20.3
- WinoGrandeLlama 3.1 Nemotron Instruct 70B by 17.584.567+17.5
- HellaSwagLlama 3.1 Nemotron Instruct 70B by 16.585.669.1+16.5
- τ²-Bench Telecom (AA run)Llama 3.1 Nemotron Instruct 70B by 14.923.18.2+14.9
- ARC-ChallengePhi 4 Mini Instruct by 14.569.283.7+14.5
- TruthfulQAPhi 4 Mini Instruct by 7.858.666.4+7.8
- Artificial Analysis Coding IndexLlama 3.1 Nemotron Instruct 70B by 710.83.8+7
- GSM8KLlama 3.1 Nemotron Instruct 70B by 2.891.488.6+2.8
- AA Intelligencetie6.96.3tie
Questions people ask
Which is better, Llama 3.1 Nemotron Instruct 70B or Phi 4 Mini Instruct?
Llama 3.1 Nemotron Instruct 70B wins four of the six areas where both have results: coding, reasoning, facts and following instructions. Phi 4 Mini Instruct wins long documents. They are level on agents.
Which is better for coding?
Llama 3.1 Nemotron Instruct 70B. It wins 2 of the 2 coding tests both models report; Phi 4 Mini Instruct wins none.
How do you compare the two?
We use the 20 benchmark tests both models have published scores on. The verdict counts the 10 tests in the eight capability areas, and a gap under one point (ten on rating-style scales) counts as a tie. The other 10 are listed but not counted, because several are summary scores or repeat a test. Each score is the one shown on the model's own page.