Skip to content

Models741 Deals30d263 Policy30d65, 14 pendingSources1,013

MMLU leaderboard — the family board.

33 TRACKED VARIANTS · 5 FIRST-CLASS BOARDS

One family, 33 tracked prints: MMLU-Pro, MMLU-Redux, MMLU-ProX and MMLU-Pro (HF Open LLM) are cited standalone and keep live boards of their own. Scoring configs and splits of the base board render as sections of this page, each under its own honest label.

● LIVE DATA150 MODELS ON THIS BOARD15 LANGUAGE BOARDS FOLD TO ONE TABLELEADER CLAUDE OPUS 4 AT 92.9

First-class boards in this familyCITED STANDALONE · EACH KEEPS ITS OWN LIVE BOARD

#BoardModels scored

02MMLUTHIS PAGE150

05MMLU-Pro (HF Open LLM)17

The board — MMLUTOP 100 OF 150 MEASURED · BEST SCORE PER MODEL, DEFAULT VARIANT

#ModelVendorScoreVLast seen

01Claude Opus 4Anthropic92.92026-06-15

02Gemini 3.1 ProGoogle92.62026-08-24

03o1OpenAI91.82026-06-04

04Claude Sonnet 4Anthropic91.52026-06-15

05DeepSeek-R1DeepSeek90.82026-06-04

06DeepSeek-V4-Pro-BaseDeepSeek90.12026-06-27

07Hunyuan Large InstructTencent89.92026-05-31

08Kimi K2 InstructMoonshot89.52026-06-15

09DeepSeek-V3DeepSeek89.42026-06-15

10MiMo V2.5 Pro BaseXiaomi89.42026-06-05

11Nemotron 3 Ultra 550B A55B BaseNVIDIA89.12026-06-05

12DeepSeek-R1-ZeroDeepSeek88.82026-05-03

13DeepSeek-V4-Flash-BaseDeepSeek88.72026-06-27

14Claude 3.5 SonnetAnthropic88.72026-09-07

15Llama 3.1 Instruct 405BMeta88.62026-08-29

16MiniMax Text 01MiniMax88.52026-06-06

17Hunyuan LargeTencent88.42026-05-31

18Kimi K2 BaseMoonshot88.22026-06-13

19GPT-4oOpenAI88.12026-06-06

20DeepSeek-V3.2-Exp-BaseDeepSeek87.82026-06-13

21DeepSeek-V3.2-BaseDeepSeek87.82026-06-27

22GLM-4.5-BaseZ.ai87.72026-06-05

23DeepSeek-V3-BaseDeepSeek87.72026-06-05

24MAI-Thinking-1Microsoft87.62026-08-14

25Hy3-preview-BaseTencent87.42026-06-05

26Seed Oss 36B InstructByteDance87.42026-06-15

27DeepSeek-V3.1-BaseDeepSeek87.42026-06-13

28Mistral Large 3 675B Base 2512Mistral87.32026-06-05

29Qwen3 235B A22BAlibaba872026-06-15

30Qwen3 30B A3B 2507 ThinkingAlibaba86.92026-06-15

31Claude 3 OpusAnthropic86.82026-09-07

32Gemini 1.5 ProGoogle86.82026-09-07

33MiMo V2 Flash BaseXiaomi86.72026-06-13

34Gemini 2.0 Flash ExpGoogle86.52026-06-06

35GPT-4OpenAI86.42026-08-12

36Llama 3.3 70B InstructMeta86.32026-05-30

37MiMo V2.5 BaseXiaomi86.32026-06-05

38Qwen3 32BAlibaba86.22026-06-15

39Qwen2.5 Instruct 72BAlibaba86.12026-06-06

40Nemotron 3 Super 120B A12BNVIDIA86.02026-08-11

41Llama 3.1 70B InstructMeta862026-05-31

42Step3.5 Flash BaseStepFun85.82026-06-05

43o1-miniOpenAI85.22026-06-04

44Phi 4Microsoft84.82026-05-30

45Qwen2 72BAlibaba84.22026-09-10

46Kimi VL A3B ThinkingMoonshot822026-06-05

47GPT-4o miniOpenAI822026-08-25

48Qwen3.5 35B A3BAlibaba81.12026-08-11

49Qwen3 14B BaseAlibaba80.42026-06-05

50DeepSeek-V2.5DeepSeek80.42026-05-31

51Granite 4.1 30BIBM80.22026-06-15

52Qwen 2.5 14BAlibaba79.92026-05-30

53Llama 3 (70B)Meta79.52026-09-10

54Ministral 3 14BMistral79.42026-06-12

55Nemotron 3.5 LightningNVIDIA78.62026-08-11

56Claude 2Anthropic78.52026-08-12

57DeepSeek-V2DeepSeek78.52026-09-10

58NVIDIA Nemotron 3 Nano 30B A3BNVIDIA78.52026-08-11

59Granite 4.1 30B BaseIBM78.42026-06-15

60Qwen2.5 VL 32B InstructAlibaba78.42026-08-25

61Claude 3 SonnetAnthropic78.32026-09-07

62Phi 3 Medium 4K InstructMicrosoft77.92026-09-21

63Gemma 4 26B A4BGoogle77.82026-08-11

64Claude 3.5 HaikuAnthropic77.62026-08-25

65Claude 1.3Anthropic772026-07-17

66Qwen2 Instruct (72B)Alibaba76.92026-05-10

67Gemma 3 27BGoogle76.92026-08-25

68Ministral 3 8BMistral76.12026-06-05

69Qwen3 8B BaseAlibaba762026-06-05

70Nova ProAmazon75.82026-05-10

71Qwen2.5 7B BaseAlibaba74.22026-05-03

72Palmyra-X-004Writer73.92026-05-10

73Granite 4.1 8BIBM73.82026-06-15

74Granite 4.1 8B BaseIBM73.62026-06-15

75Llama 3.1 8B InstructMeta732026-06-11

76Inflection-1Inflection AI72.72026-08-12

77Mistral Large 2Mistral72.52026-05-10

78Qwen3 4B 2507 InstructAlibaba72.32026-10-07

79Qwen3 4B BaseAlibaba71.32026-06-05

80Yi Large (Preview)01.AI71.22026-05-10

81GPT-4 TurboOpenAI71.12026-06-18

82Phi 3 Mini 4K InstructMicrosoft70.92026-08-29

83Ministral 3 3BMistral70.72026-06-05

84Gemini 1.5 Flash (001)Google70.32026-05-10

85Llama 3.2 Instruct 90B (Vision)Meta70.32026-05-10

86Palmyra X V3 (72B)Writer70.22026-05-10

87MoonlightMoonshot702026-06-12

88Nova LiteAmazon69.32026-06-23

89Jamba 1.5 LargeAI2168.32026-05-10

90Solar ProUpstage67.92026-05-10

91Granite 4.1 3BIBM67.02026-06-15

92Granite 3.2 8B InstructIBM66.82026-06-11

93Granite 4.0 H TinyIBM66.82026-10-07

94Granite 3.1 8B InstructIBM66.82026-06-11

95Llama 3 Instruct 8BMeta66.62026-08-29

96Granite 4.1 3B BaseIBM66.52026-06-15

97Gemma 2 27B ITGoogle66.42026-05-10

98Claude 3 HaikuAnthropic66.22026-05-10

99Phi 3 Small 8K InstructMicrosoft65.92026-05-10

100Qwen2.5 Instruct Turbo (7B)Alibaba65.82026-05-10

◆ IV — INDEPENDENTLY VERIFIED · ◆ AA — AGGREGATOR · ◆ VA — VENDOR · ◇ SA — SOURCE-ATTRIBUTED

Variants of this boardSCORING CONFIGS & SPLITS — NOT CITED STANDALONE, SO THEY LIVE HERE

MMLU-Pro test

VARIANT7 models · best score per model

#ModelVendorScoreVLast seen

01Ministral 3 3BMistral47.52026-08-12

02Gemma 4 E2BGoogle44.12026-08-12

03Qwen3 VL 2B InstructAlibaba42.82026-08-12

04North-Micro-Vision-InstructCohere30.72026-08-12

05Qwen3.5 2BAlibaba29.82026-08-12

06Phi 3.5 Vision InstructMicrosoft28.62026-08-12

07LFM2.5-VL-1.6BLiquid AI19.92026-08-12

MMLU test

VARIANT7 models · best score per model

#ModelVendorScoreVLast seen

01Gemma 4 E2BGoogle69.22026-08-12

02Ministral 3 3BMistral662026-08-12

03Qwen3 VL 2B InstructAlibaba632026-08-12

04Qwen3.5 2BAlibaba54.32026-08-12

05North-Micro-Vision-InstructCohere50.42026-08-12

06LFM2.5-VL-1.6BLiquid AI46.42026-08-12

07Phi 3.5 Vision InstructMicrosoft35.52026-08-12

MMLU Redux 5-shot

VARIANT6 models · best score per model

#ModelVendorScoreVLast seen

01Qwen3 14B BaseAlibaba83.72026-06-05

02Ministral 3 14BMistral822026-06-05

03Qwen3 8B BaseAlibaba79.42026-06-05

04Ministral 3 8BMistral79.32026-06-05

05Qwen3 4B BaseAlibaba75.92026-06-05

06Ministral 3 3BMistral73.52026-06-05

MMLU-ProX (avg en/de/fr/es/it/ja/zh/hi/pt/ko)

VARIANT5 models · best score per model

#ModelVendorScoreVLast seen

01Qwen3.5 397B A17BAlibaba86.42026-06-10

02GLM-5.1Z.ai85.82026-06-10

03Kimi K2.6Moonshot852026-06-10

04Nemotron 3 Ultra 550B A55BNVIDIA832026-06-10

05MiniMax M2.7MiniMax78.42026-06-10

MMLU by language

PER-LANGUAGE15 single-language boards, one table

LanguageLeaderScoreModels

Average over languageso388.84

Chinese (Simplified)o389.34

Frencho390.64

Hindio389.84

Indonesiano389.84

Italiano391.24

Koreano389.34

Arabico390.43

Bengalio387.83

Germano390.53

Japaneseo188.93

Portuguese (Brazil)o189.53

Swahilio185.43

Yorubao175.43

Spanisho391.12

Other tracked prints

THIN BOARDSunder 5 published models each — listed, never hidden

PrintLeaderScoreModels

MMLU (Acc.)DeepSeek-V387.14

MMLU (CoT)Llama 3.1 Instruct 405B88.64

MMLU-Pro (Acc.)DeepSeek-V364.44

MMLU Pro (CoT)Llama 3.1 Instruct 405B73.34

MMLU-Redux (Acc.)DeepSeek-V386.24

MMLU-ProX lite (IBM)Granite 4.2 30B66.63

MMLU-ProX (avg over langs)Nemotron 3 Super 120B A12B79.52

MMLU-ProX-LiteQwen3.5 2B55.22

Also tracked, no published surface rows yet: MMLU* (speech understanding)

THE FAMILY BOARD · EVERY PRINT LABELED · EVERY SCORE SOURCE-ATTRIBUTED ON ITS OWN LIVE BOARDAll leaderboards The frontier board Models by vendor