LFM2.5-1.2B-Instruct vs Llama 3.2 Instruct 1B
Wins 2 of 5 areas
Reasoning · Following instructions
Wins 2 of 5 areas
Facts · Long documents
The two are evenly matched.LFM2.5-1.2B-Instruct is better at reasoning and following instructions; Llama 3.2 Instruct 1B at facts and long documents.
Scores updated · 27 tests both models report · How we compare
Where each one wins
Tests won in each of the five areas where both have results. Each piece is one test, so a longer bar means more evidence; grey means the two scored within a point of each other.
- ReasoningHard problems that need careful thinking20LFM2.5-1.2B-Instruct2 of 3 tests · 1 tie
- Following instructionsDoing exactly what it is asked10LFM2.5-1.2B-Instruct1 of 1 test
- FactsGetting facts right instead of making them up01Llama 3.2 Instruct 1B1 of 2 tests · 1 tie
- Long documentsFinding answers in very long texts01Llama 3.2 Instruct 1B1 of 1 test
- CodingWriting and fixing software00Even0 each · 2 ties
Long documents and following instructions rest on a single test each.
What it costs
Prices per million tokens, roughly 750,000 words. The bars show the cost of a million tokens read plus a million written.
The biggest differences
The tests each model wins by the widest margin, up to three each. Scores are out of 100.
Where LFM2.5-1.2B-Instruct pulls ahead
- Follows unfamiliar, precisely checkable instructionsIFBench+21points ahead
- Graduate-level biology, physics and chemistry questionsGPQA Diamond+13points ahead
- Very hard expert questions across many subjectsHumanity's Last Exam+1.2points ahead
Where Llama 3.2 Instruct 1B pulls ahead
- Avoids making up answers it doesn't knowAA-Omniscience · Non-hallucination+18.8points ahead
Every test, side by side
All 27 tests both models report. The winning score is in its model's colour; marks a score checked independently.
ReasoningLFM2.5-1.2B-Instruct
- GPQA DiamondLFM2.5-1.2B-Instruct by 1332.619.6+13
- Humanity's Last ExamLFM2.5-1.2B-Instruct by 1.26.75.5+1.2
- CritPttie00tie
FactsLlama 3.2 Instruct 1B
- AA-Omniscience · Non-hallucinationLlama 3.2 Instruct 1B by 18.814.933.7+18.8
- AA-Omniscience · Accuracytie77tie
Long documentsLlama 3.2 Instruct 1B
- AA-LCRLlama 3.2 Instruct 1B by 6.706.7+6.7
Following instructionsLFM2.5-1.2B-Instruct
- IFBenchLFM2.5-1.2B-Instruct by 2143.822.8+21
Other results18 tests, not counted
Tests outside the eight areas. They are not counted above: several are summary scores built from other tests, or the same test under another name.
- J-MATH500LFM2.5-1.2B-Instruct by 38.65011.4+38.6
- IFEvalLFM2.5-1.2B-Instruct by 33.886.252.4+33.8
- Avg.LFM2.5-1.2B-Instruct by 31.850.118.3+31.8
- GSM8KLlama 3.2 Instruct 1B by 31.44.335.7+31.4
- BFCL v3LFM2.5-1.2B-Instruct by 27.749.121.4+27.7
- J-BFCLv3LFM2.5-1.2B-Instruct by 25.246.321.1+25.2
- J-GSM8KLFM2.5-1.2B-Instruct by 2550.225.2+25
- MMLU-ProLFM2.5-1.2B-Instruct by 23.644.420.8+23.6
- Domain AvgLFM2.5-1.2B-Instruct by 18.539.721.2+18.5
- MATH-500 (EM)Llama 3.2 Instruct 1B by 17.65.823.4+17.6
- AA-OmniscienceLlama 3.2 Instruct 1B by 17.4-72.1-54.7+17.4
- JMMLU-ProXLFM2.5-1.2B-Instruct by 15.531.415.9+15.5
- JMMLULFM2.5-1.2B-Instruct by 13.647.634+13.6
- JHumanEval+LFM2.5-1.2B-Instruct by 1128.717.7+11
- τ²-Bench Telecom (AA run)LFM2.5-1.2B-Instruct by 10.810.80+10.8
- AA Agentic IndexLFM2.5-1.2B-Instruct by 3.63.60+3.6
- JGPQAtie31.732.3tie
- AA Intelligencetie5.24.8tie
Questions people ask
Which is better, LFM2.5-1.2B-Instruct or Llama 3.2 Instruct 1B?
LFM2.5-1.2B-Instruct and Llama 3.2 Instruct 1B each win two of the five areas where both have results. LFM2.5-1.2B-Instruct wins reasoning and following instructions; Llama 3.2 Instruct 1B wins facts and long documents. They are level on coding.
Which is better for coding?
Neither. All 2 coding tests both models report are ties.
How do you compare the two?
We use the 27 benchmark tests both models have published scores on. The verdict counts the 9 tests in the eight capability areas, and a gap under one point (ten on rating-style scales) counts as a tie. The other 18 are listed but not counted, because several are summary scores or repeat a test. Each score is the one shown on the model's own page.