LFM2.5-1.2B-Instruct vs Qwen3 1.7B (instruct)
Wins 1 of 2 areas
Following instructions
Wins 1 of 2 areas
Reasoning
The two are evenly matched.LFM2.5-1.2B-Instruct is better at following instructions; Qwen3 1.7B (instruct) at reasoning.
Scores updated · 17 tests both models report · How we compare
Where each one wins
Tests won in each of the two areas where both have results. Each piece is one test, so a longer bar means more evidence; grey means the two scored within a point of each other.
Reasoning and following instructions rest on a single test each.
The biggest differences
The tests each model wins by the widest margin, up to three each. Scores are out of 100.
Where LFM2.5-1.2B-Instruct pulls ahead
- Follows unfamiliar, precisely checkable instructionsIFBench+22.5points ahead
Where Qwen3 1.7B (instruct) pulls ahead
- Graduate-level biology, physics and chemistry questionsGPQA Diamond+2.3points ahead
Every test, side by side
All 17 tests both models report. The winning score is in its model's colour; marks a score checked independently.
ReasoningQwen3 1.7B (instruct)
- GPQA DiamondQwen3 1.7B (instruct) by 2.332.634.9+2.3
Following instructionsLFM2.5-1.2B-Instruct
- IFBenchLFM2.5-1.2B-Instruct by 22.543.821.3+22.5
Other results15 tests, not counted
Tests outside the eight areas. They are not counted above: several are summary scores built from other tests, or the same test under another name.
- MATH-500 (EM)Qwen3 1.7B (instruct) by 64.65.870.4+64.6
- GSM8KQwen3 1.7B (instruct) by 29.44.333.7+29.4
- JHumanEval+Qwen3 1.7B (instruct) by 18.928.747.6+18.9
- IFEvalLFM2.5-1.2B-Instruct by 12.586.273.7+12.5
- J-MATH500Qwen3 1.7B (instruct) by 6.45056.4+6.4
- J-BFCLv3Qwen3 1.7B (instruct) by 6.246.352.5+6.2
- JGPQALFM2.5-1.2B-Instruct by 5.431.726.3+5.4
- Domain AvgQwen3 1.7B (instruct) by 5.139.744.8+5.1
- AIME25 no toolsLFM2.5-1.2B-Instruct by 4.7149.3+4.7
- J-GSM8KLFM2.5-1.2B-Instruct by 4.250.246+4.2
- BFCL v3LFM2.5-1.2B-Instruct by 2.849.146.3+2.8
- MMLU-ProLFM2.5-1.2B-Instruct by 1.544.442.9+1.5
- Avg.Qwen3 1.7B (instruct) by 1.150.151.2+1.1
- JMMLU-ProXtie31.430.8tie
- JMMLUtie47.647.7tie
Questions people ask
Which is better, LFM2.5-1.2B-Instruct or Qwen3 1.7B (instruct)?
LFM2.5-1.2B-Instruct and Qwen3 1.7B (instruct) each win one of the two areas where both have results. LFM2.5-1.2B-Instruct wins following instructions; Qwen3 1.7B (instruct) wins reasoning.
How do you compare the two?
We use the 17 benchmark tests both models have published scores on. The verdict counts the 2 tests in the eight capability areas, and a gap under one point (ten on rating-style scales) counts as a tie. The other 15 are listed but not counted, because several are summary scores or repeat a test. Each score is the one shown on the model's own page.