Across 19 shared benchmarks, Mistral Large 3 675B Base 2512 scores higher on 5 and NVIDIA-Nemotron-3-Ultra-550B-A55B-Base-BF16 on 14. The widest gap is RULER 256K, where NVIDIA-Nemotron-3-Ultra-550B-A55B-Base-BF16 scores 86.2 against 35.5.
| Benchmark | Mistral Large 3 675B Base 2512 | NVIDIA-Nemotron-3-Ultra-550B-A55B-Base-BF16 |
|---|---|---|
| AGIEval-En | 69.3 | 78.7 |
| arc_challenge | 97.3 | 97.3 |
| GPQA Diamond | 43.9 | 50 |
| GSM8K | 91.2 | 88.1 |
| hellaswag | 88.9 | 90.5 |
| humaneval | 66.7 | 83.8 |
| MATH | 62.9 | 82 |
| MBPP-Sanitized | 84.1 | 86 |
| mgsm | 82.9 | 87.7 |
| mmlu | 87.3 | 89.1 |
| MMLU Global Lite | 87.3 | 90.1 |
| MMLU-Pro | 67.4 | 79.1 |
| OpenBookQA | 51.4 | 48.6 |
| PIQA | 84.8 | 83.8 |
| RACE | 93.3 | 92.2 |
| RULER 128K | 55.8 | 92.5 |
| RULER 256K | 35.5 | 86.2 |
| RULER 64K | 90.1 | 95.3 |
| winogrande | 82.1 | 79.3 |
Best tracked score per model per benchmark (default configuration; source-attributed). ↓ marks lower-is-better metrics. Open either model for its full surface, provenance and pricing.