How many tracked prints does the HumanEval family have?

HumanEval leaderboard — the family board.

4 TRACKED VARIANTS · 3 FIRST-CLASS BOARDS

One family, 4 tracked prints: HumanEval+ and HumanEval-V are cited standalone and keep live boards of their own. Scoring configs and splits of the base board render as sections of this page, each under its own honest label.

● LIVE DATA104 MODELS ON THIS BOARDLEADER KIMI K2 INSTRUCT AT 94.5

First-class boards in this familyCITED STANDALONE · EACH KEEPS ITS OWN LIVE BOARD

#BoardModels scored

01HumanEvalTHIS PAGE104

02HumanEval+39

03HumanEval-V5

The board — HumanEvalTOP 100 OF 104 MEASURED · BEST SCORE PER MODEL, DEFAULT VARIANT

#ModelVendorScoreVLast seen

01Kimi K2 InstructMoonshot94.52026-09-01

02Claude 3.5 SonnetAnthropic93.72026-08-23

03GPT-5OpenAI93.42026-08-23

04Qwen 2.5 Coder 32B InstructAlibaba92.72026-08-23

05o1-miniOpenAI92.42026-08-23

06Sarvam 30BSarvam AI92.12026-08-23

07DeepSeek-V3DeepSeek92.12026-06-06

08Mistral Large 2Mistral922026-08-23

09Qwen2.5 VL 32B InstructAlibaba91.52026-08-26

10GPT-4oOpenAI90.62026-08-23

11Hunyuan Large InstructTencent902026-05-31

12Granite 3.3 8B InstructIBM89.72026-08-23

13Granite 3.1 8B InstructIBM89.62026-06-11

14Gemini 2.0 Flash ExpGoogle89.62026-06-06

15Granite 3.2 8B InstructIBM89.32026-06-11

16Nova ProAmazon892026-09-01

17Llama 3.1 Instruct 405BMeta892026-09-11

18DeepSeek-V2.5DeepSeek892026-08-23

19Granite 4.1 30BIBM88.42026-06-15

20Grok 2SpaceXAI88.42026-08-23

21Llama 3.3 70B InstructMeta88.42026-08-23

22Qwen2.5 Coder Instruct 7BAlibaba88.42026-06-05

23o1OpenAI88.12026-08-23

24Claude 3.5 HaikuAnthropic88.12026-08-25

25GPT-4.5 PreviewOpenAI882026-08-23

26Gemma 3 27BGoogle87.82026-08-25

27GPT-4o miniOpenAI87.22026-08-25

28GPT-4 TurboOpenAI87.12026-08-23

29MiniMax Text 01MiniMax86.92026-06-06

30Gemini 1.5 ProGoogle86.62026-08-23

31Qwen2.5 Instruct 72BAlibaba86.62026-08-23

32Qwen2 Instruct (72B)Alibaba862026-08-23

33Grok 2 MiniSpaceXAI85.72026-08-23

34Gemma 3 12BGoogle85.42026-08-23

35Nova LiteAmazon85.42026-09-01

36Granite 4.1 8BIBM85.42026-06-15

37Llama 3.1 8B InstructMeta85.32026-08-23

38Claude 3 OpusAnthropic84.92026-08-23

39Kimi K2 BaseMoonshot84.82026-06-05

40Qwen3 8BAlibaba84.82026-06-05

41MAI-Thinking-1Microsoft84.22026-08-14

42Nemotron 3 Ultra 550B A55B BaseNVIDIA83.82026-06-05

43Seed Coder 8B InstructByteDance82.92026-06-05

44Phi 4Microsoft82.62026-08-23

45Yi Coder 9B Chat01.AI82.32026-06-05

46Granite 4.1 3BIBM81.72026-06-15

47Nova MicroAmazon81.12026-09-01

48Step3.5 Flash BaseStepFun81.12026-06-05

49Granite 3.3 2B InstructIBM80.52026-06-11

50Llama 3.1 70B InstructMeta80.52026-08-23

51Nemotron 3 Super 120B A12BNVIDIA80.52026-08-11

52Granite 3.2 2B InstructIBM80.12026-06-11

53Qwen2 7B InstructAlibaba79.92026-08-23

54DeepSeek-R1-Distill-Qwen-7BDeepSeek79.92026-06-11

55GLM-4.5-BaseZ.ai79.82026-06-05

56Granite 3.1 2B InstructIBM79.52026-06-11

57Stable-DiffCoder-8B-BaseByteDance79.32026-06-05

58Qwen2.5 Omni 7BAlibaba78.72026-08-23

59Nemotron 3.5 LightningNVIDIA77.42026-08-11

60MiMo V2 Flash BaseXiaomi77.42026-06-05

61DeepSeek-V4-Pro-BaseDeepSeek76.82026-09-10

62Granite 4.1 3B BaseIBM76.22026-06-15

63Claude 3 HaikuAnthropic75.92026-08-23

64NVIDIA Nemotron 3 Nano 30B A3BNVIDIA752026-08-11

65DeepSeek-Coder-6.7B-InstructDeepSeek74.42026-06-05

66Claude 3 SonnetAnthropic732026-08-23

67DeepSeek-V3.1-BaseDeepSeek72.52026-06-05

68Qwen 2.5 14BAlibaba72.12026-05-30

69Hunyuan LargeTencent71.42026-05-31

70Gemma 3 4BGoogle71.32026-08-23

71Claude 2Anthropic71.22026-08-12

72Phi 3.5 MoE InstructMicrosoft70.72026-08-23

73DeepSeek-V4-Flash-BaseDeepSeek69.52026-09-10

74Granite 4.1 8B BaseIBM68.32026-06-15

75Phi 3 Medium 4K InstructMicrosoft67.82026-05-30

76DeepSeek-V3.2-Exp-BaseDeepSeek67.72026-06-05

77Granite 4.1 30B BaseIBM67.72026-06-15

78DeepSeek-R1-Distill-Llama-8BDeepSeek67.52026-06-11

79GPT-4OpenAI672026-08-23

80Mistral Large 3 675B Base 2512Mistral66.72026-06-05

81Qwen3.5 35B A3BAlibaba66.52026-08-11

82Qwen2 72BAlibaba64.62026-09-10

83Phi 3.5 Mini InstructMicrosoft62.82026-08-23

84DeepSeek-V3.2-BaseDeepSeek62.82026-06-27

85Llama 3 Instruct 8BMeta60.42026-08-29

86Phi 3 Mini 4K InstructMicrosoft57.32026-08-29

87Gemma 2 27B ITGoogle51.82026-08-23

88Gemma 4 26B A4BGoogle502026-08-11

89DeepSeek-V2DeepSeek48.82026-09-10

90Llama 3 (70B)Meta48.22026-09-10

91MoonlightMoonshot48.12026-06-12

92DeepSeek-Coder-6.7B-BaseDeepSeek47.62026-06-05

93Qwen2.5 3BAlibaba42.12026-06-05

94Gemma 3 1B InstructGoogle41.52026-08-23

95Gemma 2 Instruct (9B)Google40.22026-08-23

96Gemma (7B)Google37.22026-08-29

97Inflection-1Inflection AI35.42026-08-12

98Mistral 7BMistral29.32026-08-29

99Llama 3.2 3B InstructMeta282026-06-05

100LFM2.5-8B-A1B-DSparkLiquid AI7.02026-08-21

◆ IV — INDEPENDENTLY VERIFIED · ◆ AA — AGGREGATOR · ◆ VA — VENDOR · ◇ SA — SOURCE-ATTRIBUTED

Variants of this boardSCORING CONFIGS & SPLITS — NOT CITED STANDALONE, SO THEY LIVE HERE

HumanEval-Mul (Pass@1)

VARIANT7 models · best score per model

#ModelVendorScoreVLast seen

01DeepSeek-V3DeepSeek82.62026-08-23

02Claude 3.5 SonnetAnthropic81.72026-05-03

03GPT-4oOpenAI80.52026-05-03

04DeepSeek-V2.5DeepSeek77.42026-08-23

05Qwen2.5 Instruct 72BAlibaba77.32026-05-03

06Llama 3.1 Instruct 405BMeta77.22026-05-03

07DeepSeek-V2DeepSeek69.32026-05-03

THE FAMILY BOARD · EVERY PRINT LABELED · EVERY SCORE SOURCE-ATTRIBUTED ON ITS OWN LIVE BOARDAll leaderboards The frontier board Models by vendor