Nemotron 3 Ultra 550B A55B wins more areas, narrowly.Qwen3.5 397B A17B is better at coding.
Both rank among the ten best models we track in following instructions.
Scores updated · 53 tests both models report · How we compare
Where each one wins
Tests won in each of the seven areas where both have results. Each piece is one test, so a longer bar means more evidence; grey means the two scored within a point of each other.
- FactsGetting facts right instead of making them up21Nemotron 3 Ultra 550B A55B2 of 3 tests
- MathCompetition and research-level math21Nemotron 3 Ultra 550B A55B2 of 3 tests
- CodingWriting and fixing software25Qwen3.5 397B A17B5 of 7 tests
- AgentsCarrying out multi-step tasks on its own11Even1 each · 2 ties
- ReasoningHard problems that need careful thinking11Even1 each · 1 tie
- Long documentsFinding answers in very long texts11Even1 each
- Following instructionsDoing exactly what it is asked11Even1 each
What it costs
Prices per million tokens, roughly 750,000 words. The bars show the cost of a million tokens read plus a million written.
Nemotron 3 Ultra 550B A55B costs 26% less for the same work.
The biggest differences
The three tests each model wins by the widest margin. Scores are out of 100.
Where Nemotron 3 Ultra 550B A55B pulls ahead
- Avoids making up answers it doesn't knowAA-Omniscience · Non-hallucination+59.2points ahead
- Olympiad math problems with short answersIMOAnswerBench+11.4points ahead
- Real work tasks from 44 professionsGDPVal+11.1points ahead
Where Qwen3.5 397B A17B pulls ahead
- Finds hard-to-locate facts by browsing the webBrowseComp+24.6points ahead
- Harvard-MIT high-school math contest problemsHMMT Feb. 2026+9.1points ahead
- Answers hard knowledge questions correctlyAA-Omniscience · Accuracy+8.2points ahead
Every test, side by side
All 53 tests both models report. The winning score is in its model's colour; marks a score checked independently.
CodingQwen3.5 397B A17B
- SWE-bench VerifiedQwen3.5 397B A17B by 5.770.776.4+5.7
- LiveCodeBench v6Nemotron 3 Ultra 550B A55B by 5.48983.6+5.4
- SciCodeQwen3.5 397B A17B by 4.540.344.8+4.5
- Terminal-Bench HardQwen3.5 397B A17B by 4.536.440.9+4.5
- Terminal-Bench 2.1Nemotron 3 Ultra 550B A55B by 2.653.951.3+2.6
- SWE-bench MultilingualQwen3.5 397B A17B by 1.667.769.3+1.6
- SWE-bench ProQwen3.5 397B A17B by 1.249.750.9+1.2
AgentsEven
- BrowseCompQwen3.5 397B A17B by 24.644.469+24.6
- GDPValNemotron 3 Ultra 550B A55B by 11.125.714.6+11.1
- τ-Bench V3 · Bankingtie14.213.4tie
- Terminal-Bench 4.0tie0.50tie
ReasoningEven
- GPQA DiamondQwen3.5 397B A17B by 2.686.789.3+2.6
- CritPtNemotron 3 Ultra 550B A55B by 1.43.11.7+1.4
- Humanity's Last Examtie28.429tie
FactsNemotron 3 Ultra 550B A55B
- AA-Omniscience · Non-hallucinationNemotron 3 Ultra 550B A55B by 59.270.311.1+59.2
- AA-Omniscience · AccuracyQwen3.5 397B A17B by 8.222.630.8+8.2
- SimpleQA VerifiedNemotron 3 Ultra 550B A55B by 6.432.426+6.4
MathNemotron 3 Ultra 550B A55B
- IMOAnswerBenchNemotron 3 Ultra 550B A55B by 11.492.380.9+11.4
- HMMT Feb. 2026Qwen3.5 397B A17B by 9.178.887.9+9.1
- AIME 2026Nemotron 3 Ultra 550B A55B by 2.994.291.3+2.9
Long documentsEven
- AA-LCRNemotron 3 Ultra 550B A55B by 279.377.3+2
- LongBench v2Qwen3.5 397B A17B by 1.361.963.2+1.3
Following instructionsEven
- Multi-ChallengeQwen3.5 397B A17B by 3.863.867.6+3.8
- IFBenchNemotron 3 Ultra 550B A55B by 2.681.478.8+2.6
Other results29 tests, not counted
Tests outside the eight areas. They are not counted above: several are summary scores built from other tests, or the same test under another name.
- AA-OmniscienceNemotron 3 Ultra 550B A55B by 30.3-0.4-30.8+30.3
- Apex-Shortlist (with tools)Nemotron 3 Ultra 550B A55B by 24.484.860.4+24.4
- GDPval-AA v2Nemotron 3 Ultra 550B A55B by 202 rating points1164962+202 rating
- BrowseComp (context management)Qwen3.5 397B A17B by 15.66378.6+15.6
- Apex-Shortlist (no tools)Nemotron 3 Ultra 550B A55B by 13.574.961.4+13.5
- IOI 2025Nemotron 3 Ultra 550B A55B by 123.4 rating points564.7441.3+123.4 rating
- τ²-Bench Telecom (AA run)Qwen3.5 397B A17B by 12.383.395.6+12.3
- AA Agentic IndexNemotron 3 Ultra 550B A55B by 11.121.710.6+11.1
- HLE (with tools)Qwen3.5 397B A17B by 10.937.448.3+10.9
- IMOAnswerBench (with tools)Nemotron 3 Ultra 550B A55B by 7.892.384.5+7.8
- Toolathlon VerifiedQwen3.5 397B A17B by 6.434.340.7+6.4
- Vals.ai Financial Agent 1.1 - with web searchQwen3.5 397B A17B by 5.353.759+5.3
- AA IntelligenceNemotron 3 Ultra 550B A55B by 4.522.918.4+4.5
- SciCode (subtask)Qwen3.5 397B A17B by 4.543.548+4.5
- SWEBench Pro PublicQwen3.5 397B A17B by 4.546.450.9+4.5
- IFBench (prompt loose) (Tools-allowed)Nemotron 3 Ultra 550B A55B by 3.581.778.2+3.5
- TauBench V3 - AirlineNemotron 3 Ultra 550B A55B by 3.58076.5+3.5
- MMLU-ProX (avg en/de/fr/es/it/ja/zh/hi/pt/ko) (Tools-allowed)Qwen3.5 397B A17B by 3.48386.4+3.4
- PinchBenchNemotron 3 Ultra 550B A55B by 3.49086.6+3.4
- ProfBench (Search)Nemotron 3 Ultra 550B A55B by 3.456.453+3.4
- MMLU-ProXQwen3.5 397B A17B by 1.78384.7+1.7
- Vals.ai Financial Agent 1.1 - without web searchQwen3.5 397B A17B by 1.260.161.3+1.2
- Artificial Analysis Coding IndexNemotron 3 Ultra 550B A55B by 1.149.348.2+1.1
- CritPt (no tools)Nemotron 3 Ultra 550B A55B by 13.42.4+1
- MMLU-ProQwen3.5 397B A17B by 186.887.8+1
- StrongREJECTtie98.799.4tie
- TauBench V3 - Averagetie70.371tie
- τ³-Bench Bankingtie13.813.4tie
- TauBench V3 - Retailtie88.488.5tie
Questions people ask
Which is better, Nemotron 3 Ultra 550B A55B or Qwen3.5 397B A17B?
Nemotron 3 Ultra 550B A55B wins two of the seven areas where both have results: facts and math. Qwen3.5 397B A17B wins coding. They are level on agents, reasoning, long documents and following instructions.
Which is better for coding?
Qwen3.5 397B A17B. It wins 5 of the 7 coding tests both models report; Nemotron 3 Ultra 550B A55B wins 2.
Which is cheaper?
Nemotron 3 Ultra 550B A55B costs $0.60 per million input tokens and $2.50 per million output tokens; Qwen3.5 397B A17B costs $0.60 and $3.60. That makes Nemotron 3 Ultra 550B A55B about 26% cheaper for the same work.
How do you compare the two?
We use the 53 benchmark tests both models have published scores on. The verdict counts the 24 tests in the eight capability areas, and a gap under one point (ten on rating-style scales) counts as a tie. The other 29 are listed but not counted, because several are summary scores or repeat a test. Each score is the one shown on the model's own page.