LFM2-8B-A1B (Non-Reasoning) vs Qwen3 4B 2507 Instruct
Wins 0 of 4 areas
—
Wins 4 of 4 areas
Coding · Reasoning · Facts · Following instructions
Qwen3 4B 2507 Instruct is the stronger all-rounder.
Scores updated · 20 tests both models report · How we compare
Where each one wins
Tests won in each of the four areas where both have results. Each piece is one test, so a longer bar means more evidence; grey means the two scored within a point of each other.
- CodingWriting and fixing software03Qwen3 4B 2507 Instruct3 of 3 tests
- FactsGetting facts right instead of making them up02Qwen3 4B 2507 Instruct2 of 2 tests
- ReasoningHard problems that need careful thinking01Qwen3 4B 2507 Instruct1 of 3 tests · 2 ties
- Following instructionsDoing exactly what it is asked01Qwen3 4B 2507 Instruct1 of 1 test
Following instructions rests on a single test.
What it costs
Prices per million tokens, roughly 750,000 words. The bars show the cost of a million tokens read plus a million written.
The biggest differences
The tests each model wins by the widest margin, up to three each. Scores are out of 100.
Where Qwen3 4B 2507 Instruct pulls ahead
- Recent programming contest problemsLiveCodeBench v6+27.7points ahead
- Graduate-level biology, physics and chemistry questionsGPQA Diamond+17.3points ahead
- Code for real scientific research problemsSciCode+11.3points ahead
Where LFM2-8B-A1B (Non-Reasoning) pulls ahead
No clear win on a test scored out of 100.
Every test, side by side
All 20 tests both models report. The winning score is in its model's colour; marks a score checked independently.
CodingQwen3 4B 2507 Instruct
- LiveCodeBench v6Qwen3 4B 2507 Instruct by 27.72148.7+27.7
- SciCodeQwen3 4B 2507 Instruct by 11.36.818.1+11.3
- Terminal-Bench HardQwen3 4B 2507 Instruct by 4.504.5+4.5
ReasoningQwen3 4B 2507 Instruct
- GPQA DiamondQwen3 4B 2507 Instruct by 17.334.451.7+17.3
- Humanity's Last Examtie4.94.5tie
- CritPttie00tie
FactsQwen3 4B 2507 Instruct
- AA-Omniscience · Non-hallucinationQwen3 4B 2507 Instruct by 10.67.718.3+10.6
- AA-Omniscience · AccuracyQwen3 4B 2507 Instruct by 2.78.210.9+2.7
Following instructionsQwen3 4B 2507 Instruct
- IFBenchQwen3 4B 2507 Instruct by 7.226.333.5+7.2
Other results11 tests, not counted
Tests outside the eight areas. They are not counted above: several are summary scores built from other tests, or the same test under another name.
- LCB v5Qwen3 4B 2507 Instruct by 29.421.450.8+29.4
- τ²-Bench Telecom (AA run)Qwen3 4B 2507 Instruct by 16.110.526.6+16.1
- MMLU-ProQwen3 4B 2507 Instruct by 14.937.452.3+14.9
- AA-OmniscienceQwen3 4B 2507 Instruct by 14.6-76.5-61.9+14.6
- HumanEval+Qwen3 4B 2507 Instruct by 12.869.582.3+12.8
- MATH-500 (EM)Qwen3 4B 2507 Instruct by 11.474.285.6+11.4
- MGSMQwen3 4B 2507 Instruct by 9.472.481.8+9.4
- IFEvalQwen3 4B 2507 Instruct by 877.685.6+8
- MMLUQwen3 4B 2507 Instruct by 7.564.872.3+7.5
- MMMLUQwen3 4B 2507 Instruct by 5.455.360.7+5.4
- AA IntelligenceQwen3 4B 2507 Instruct by 1.84.96.7+1.8
Questions people ask
Which is better, LFM2-8B-A1B (Non-Reasoning) or Qwen3 4B 2507 Instruct?
Qwen3 4B 2507 Instruct wins all four areas where both have results: coding, reasoning, facts and following instructions. LFM2-8B-A1B (Non-Reasoning) wins none.
Which is better for coding?
Qwen3 4B 2507 Instruct. It wins 3 of the 3 coding tests both models report; LFM2-8B-A1B (Non-Reasoning) wins none.
How do you compare the two?
We use the 20 benchmark tests both models have published scores on. The verdict counts the 9 tests in the eight capability areas, and a gap under one point (ten on rating-style scales) counts as a tie. The other 11 are listed but not counted, because several are summary scores or repeat a test. Each score is the one shown on the model's own page.