Scores updated · 6 tests both models report · How we compare
Every test, side by side
All 6 tests both models report. The winning score is in its model's colour; marks a score checked independently.
Other results6 tests
- MMLUSolar Pro by 10.457.567.9+10.4
- GSM8KSolar Pro by 10.376.887.1+10.3
- NarrativeQASolar Pro by 9.965.375.3+9.9
- OpenBookQASolar Pro by 9.482.892.2+9.4
- naturalquestions_closedbookArctic Instruct by 9.33929.7+9.3
- MATHSolar Pro by 4.851.956.7+4.8
Questions people ask
How do you compare the two?
We use the 6 benchmark tests both models have published scores on. None of them is in the eight capability areas we count, so this page lists them without a verdict. Each score is the one shown on the model's own page.