TML-d24-varloop-research wins more areas, narrowly.
Scores updated · 7 tests both models report · How we compare
Where each one wins
Tests won in each of the two areas where both have results. Each piece is one test, so a longer bar means more evidence; grey means the two scored within a point of each other.
Images and charts rests on a single test.
What it costs
Prices per million tokens, roughly 750,000 words. The bars show the cost of a million tokens read plus a million written.
The biggest differences
The tests each model wins by the widest margin, up to three each. Scores are out of 100.
Where TML-d24-varloop-research pulls ahead
- Fixes real GitHub issues in Python projectsSWE-bench Verified+33.5points ahead
- Fixes real GitHub issues in many programming languagesSWE-bench Multilingual+29points ahead
Where Gemma 4 31B pulls ahead
No clear win on a test scored out of 100.
Every test, side by side
All 7 tests both models report. The winning score is in its model's colour; marks a score checked independently.
CodingTML-d24-varloop-research
- SWE-bench VerifiedTML-d24-varloop-research by 33.546.179.6+33.5
- SWE-bench MultilingualTML-d24-varloop-research by 294372+29
Other results4 tests, not counted
Tests outside the eight areas. They are not counted above: several are summary scores built from other tests, or the same test under another name.
- SWEBench Pro v2 (Hard)TML-d24-varloop-research by 64.214.278.4+64.2
- JobBenchTML-d24-varloop-research by 55.18.263.3+55.1
- GDP (Surge AI)TML-d24-varloop-research by 14620+14
- OmniDocBench v1.6TML-d24-varloop-research by 1082.592.5+10
Questions people ask
Which is better, Gemma 4 31B or TML-d24-varloop-research?
TML-d24-varloop-research wins one of the two areas where both have results: coding. Gemma 4 31B wins none. They are level on images and charts.
Which is better for coding?
TML-d24-varloop-research. It wins 2 of the 2 coding tests both models report; Gemma 4 31B wins none.
How do you compare the two?
We use the 7 benchmark tests both models have published scores on. The verdict counts the 3 tests in the eight capability areas, and a gap under one point (ten on rating-style scales) counts as a tie. The other 4 are listed but not counted, because several are summary scores or repeat a test. Each score is the one shown on the model's own page.