Gemma 4 31B vs GPT-5.4
Wins 1 of 8 areas
Following instructions
Wins 7 of 8 areas
Coding · Agents · Reasoning · Facts · Images and charts · Math · Long documents
GPT-5.4 is the stronger all-rounder.Gemma 4 31B is cheaper and better at following instructions.
Scores updated · 43 tests both models report · How we compare
Where each one wins
Tests won in each of the eight areas we test. Each piece is one test, so a longer bar means more evidence; grey means the two scored within a point of each other.
- CodingWriting and fixing software06GPT-5.46 of 6 tests
- ReasoningHard problems that need careful thinking03GPT-5.43 of 3 tests
- Images and chartsUnderstanding pictures, charts and video03GPT-5.43 of 3 tests
- MathCompetition and research-level math03GPT-5.43 of 3 tests
- AgentsCarrying out multi-step tasks on its own13GPT-5.43 of 4 tests
- FactsGetting facts right instead of making them up12GPT-5.42 of 4 tests · 1 tie
- Long documentsFinding answers in very long texts01GPT-5.41 of 1 test
- Following instructionsDoing exactly what it is asked10Gemma 4 31B1 of 1 test
Long documents and following instructions rest on a single test each.
What it costs
Prices per million tokens, roughly 750,000 words. The bars show the cost of a million tokens read plus a million written.
Gemma 4 31B costs 97% less for the same work.
The biggest differences
The three tests each model wins by the widest margin. Scores are out of 100.
Where GPT-5.4 pulls ahead
- Command-line tasks in a real terminalTerminal-Bench 2.1+34.9points ahead
- Short factual questions, answered correctlySimpleQA Verified+34.7points ahead
- Real work tasks from 44 professionsGDPVal+31.3points ahead
Where Gemma 4 31B pulls ahead
- Avoids making up answers it doesn't knowAA-Omniscience · Non-hallucination+6.7points ahead
- Finds the root cause of IT system incidentsAA IT-Bench SRE+2.8points ahead
- Follows unfamiliar, precisely checkable instructionsIFBench+1.7points ahead
Every test, side by side
All 43 tests both models report. The winning score is in its model's colour; marks a score checked independently.
CodingGPT-5.4
- Terminal-Bench 2.1GPT-5.4 by 34.943.478.3+34.9
- SWE-bench ProGPT-5.4 by 2235.757.7+22
- Terminal-Bench HardGPT-5.4 by 21.236.457.6+21.2
- SWE-bench MultilingualGPT-5.4 by 2051.771.7+20
- SciCodeGPT-5.4 by 11.145.556.6+11.1
- LMArena · WebDevGPT-5.4 by 100 rating points13651465+100 rating
AgentsGPT-5.4
- GDPValGPT-5.4 by 31.36.137.4+31.3
- τ-Bench V3 · BankingGPT-5.4 by 24.814.839.6+24.8
- MCP AtlasGPT-5.4 by 1057.267.2+10
- AA IT-Bench SREGemma 4 31B by 2.837.334.5+2.8
ReasoningGPT-5.4
- CritPtGPT-5.4 by 221.423.4+22
- Humanity's Last ExamGPT-5.4 by 20.123.643.7+20.1
- GPQA DiamondGPT-5.4 by 6.385.792+6.3
FactsGPT-5.4
- SimpleQA VerifiedGPT-5.4 by 34.710.445.1+34.7
- AA-Omniscience · AccuracyGPT-5.4 by 30.92050.9+30.9
- AA-Omniscience · Non-hallucinationGemma 4 31B by 6.7158.3+6.7
- Vectara HHEM hallucination ratelower is bettertie7.47tie
Images and chartsGPT-5.4
- CharXiv (RQ)GPT-5.4 by 14.967.982.8+14.9
- MMMU-ProGPT-5.4 by 573.478.4+5
- LMArena · VisionGPT-5.4 by 25 rating points12771302+25 rating
MathGPT-5.4
- HMMT Feb. 2026GPT-5.4 by 20.577.297.7+20.5
- IMOAnswerBenchGPT-5.4 by 16.974.591.4+16.9
- AIME 2026GPT-5.4 by 1089.299.2+10
Following instructionsGemma 4 31B
- IFBenchGemma 4 31B by 1.775.673.9+1.7
Other results18 tests, not counted
Tests outside the eight areas. They are not counted above: several are summary scores built from other tests, or the same test under another name.
- AA-OmniscienceGPT-5.4 by 53.7-47.95.8+53.7
- MCPMarkGPT-5.4 by 44.418.162.5+44.4
- AA Agentic IndexGPT-5.4 by 37.56.744.2+37.5
- Tool-DecathlonGPT-5.4 by 33.421.254.6+33.4
- Terminal-Bench 2.0GPT-5.4 by 32.242.975.1+32.2
- Artificial Analysis Coding IndexGPT-5.4 by 27.743.471.1+27.7
- τ²-Bench Telecom (AA run)GPT-5.4 by 27.259.987.1+27.2
- NL2RepoGPT-5.4 by 25.815.541.3+25.8
- HLE (with tools)GPT-5.4 by 25.626.552.1+25.6
- AA IntelligenceGPT-5.4 by 24.314.739+24.3
- OmniDocBench 1.5GPT-5.4 by 980.189.1+9
- HMMT Nov. 2025GPT-5.4 by 8.387.595.8+8.3
- MATH-VisionGPT-5.4 by 6.485.692+6.4
- vectara_avg_summary_lengthGPT-5.4 by 5.975.881.7+5.9
- τ³-BenchGPT-5.4 by 5.467.572.9+5.4
- MMLU-ProGPT-5.4 by 2.385.287.5+2.3
- vectara_factual_consistencytie92.693tie
- vectara_answer_ratetie10099.9tie
Questions people ask
Which is better, Gemma 4 31B or GPT-5.4?
GPT-5.4 wins seven of the eight areas we test: coding, agents, reasoning, facts, images and charts, math and long documents. Gemma 4 31B wins following instructions, and costs 97% less.
Which is better for coding?
GPT-5.4. It wins 6 of the 6 coding tests both models report; Gemma 4 31B wins none.
Which is cheaper?
Gemma 4 31B costs $0.17 per million input tokens and $0.40 per million output tokens; GPT-5.4 costs $2.50 and $15.00. That makes Gemma 4 31B about 97% cheaper for the same work.
How do you compare the two?
We use the 43 benchmark tests both models have published scores on. The verdict counts the 25 tests in the eight capability areas, and a gap under one point (ten on rating-style scales) counts as a tie. The other 18 are listed but not counted, because several are summary scores or repeat a test. Each score is the one shown on the model's own page.