VECTOR WIREAI INTELLIGENCE
⌘K
NVDA$1,847+3.2%MSFT$512+1.1%GOOGL$199-0.4%META$728+2.7%AMD$184-1.2%TSM$212+0.6%PLTR$98+4.1%AI IDX4,821+1.9%
PKT
SEEDRefresh Models —Deals —Regulatory —Sources —

OpenBookQA

78 models tracked

OpenBookQA — Superset of 'openbookqa' (adds Nemotron card on same HELM Lite base), same min/max.

#ModelVendorBest scoreRunsLast seen
1Claude 3.5 SonnetAnthropic97.212026-07-02
2GPT-4 TurboOpenAI9722026-07-02
3GPT-4oOpenAI96.822026-07-02
4Qwen2.5 Instruct 72BAlibaba96.212026-07-02
5GPT4OpenAI9622026-07-02
6Nova Pro (Non-Reasoning)Amazon9622026-07-02
7Claude 3 OpusAnthropic95.612026-07-02
8DeepSeek-V3DeepSeek95.412026-07-02
9Qwen2 Instruct (72B)Alibaba95.412026-07-02
10Gemini 1.5 ProGoogle95.222026-07-02
11Jamba 1.5 LargeAI2194.812026-07-02
12Gemini 2.0 Flash (experimental)Google94.612026-07-02
13Yi Large (Preview)01.AI94.612026-07-02
14Gemini 2.0 Flash ExpGoogle94.612026-05-10
15Llama 3.2 Vision Instruct Turbo (90B)Meta94.212026-07-02
16Llama 3.1 Instruct 405BMeta9422026-07-02
17Palmyra X V3 (72B)Writer93.812026-07-02
18Llama 3.1 Instruct Turbo (70B)Meta93.812026-07-02
19Llama 3 (70B)Meta93.412026-07-02
20Mistral Large 2Mistral93.212026-07-02
21Amazon Nova LiteAmazon92.812026-07-02
22Gemini 1.5 Flash (001)Google92.822026-07-02
23Llama 3.3 Instruct Turbo (70B)Meta92.812026-07-02
24Palmyra-X-004Writer92.612026-07-02
25Solar ProUpstage92.212026-07-02
26GPT-4o miniOpenAI9212026-07-02
27Claude 3 SonnetAnthropic91.812026-07-02
28Gemma 2 Instruct (27B)Google91.812026-07-02
29Phi 3 Medium 4K InstructMicrosoft91.622026-07-02
30Phi 3 Small 8K InstructMicrosoft91.212026-07-02
31DBRX InstructDatabricks9112026-07-02
32Gemma 2 Instruct (9B)Google9112026-07-02
33Phi 3.5 MoE InstructMicrosoft89.612026-08-23
34Mistral LargeMistral89.412026-07-02
35Jamba 1.5 MiniAI218912026-07-02
36Nova Micro (Non-Reasoning)Amazon88.822026-07-02
37DeepSeek-ChatDeepSeek8812026-07-02
38Palmyra X V2 (33B)Writer87.812026-07-02
39Qwen2.5 Instruct Turbo (7B)Alibaba86.212026-07-02
40Mistral Small (2402)Mistral86.212026-07-02
41Claude 3.5 HaikuAnthropic85.422026-07-02
42Claude 3 HaikuAnthropic83.812026-07-02
43Mistral Medium (Non-Reasoning)Mistral8312026-07-02
44Command R+ (Apr '24)Cohere82.812026-07-02
45Arctic InstructSnowflake82.812026-07-02
46Mistral NemoMistral82.212026-07-02
47Gemma (7B)Google80.812026-07-02
48Phi 2Microsoft79.812026-07-02
49Jamba InstructAI2179.612026-07-02
50Phi 4 Mini InstructMicrosoft79.212026-08-23
51Phi 3.5 Mini InstructMicrosoft79.212026-08-23
52Mistral Instruct V0.3 (7B)Mistral7912026-07-02
53Command RCohere78.212026-07-02
54Mistral V0.1 (7B)Mistral77.612026-07-02
55CommandCohere77.412026-07-02
56Llama 3 Instruct 8BMeta76.612026-07-02
57Llama (65B)Meta75.412026-07-02
58Llama 3.1 Instruct Turbo (8B)Meta7412026-07-02
59Llama 3.2 Vision Instruct Turbo (11B)Meta72.412026-07-02
60Jurassic 2 Jumbo (178B)AI2168.812026-07-02
61Jurassic 2 Grande (17B)AI2161.412026-07-02
62Mistral Large 3 675B Base 2512Mistral51.412026-06-05
63Kimi K2 BaseMoonshot50.812026-06-05
64GLM-4.5-BaseZ.ai49.612026-06-05
65Gemma 4 26B A4BGoogle48.812026-08-11
66Nemotron 3 Super 120B A12BNVIDIA48.612026-08-11
67Nemotron 3 Ultra 550B A55B BaseNVIDIA48.612026-06-05
68NVIDIA-Nemotron-3-Ultra-550B-A55B-Base-BF16NVIDIA48.612026-08-24
69DeepSeek-V3.2-Exp-BaseDeepSeek48.212026-06-05
70Nemotron 3.5 LightningNVIDIA47.612026-08-11
71NVIDIA-Nemotron-3.5-Lightning-30B-A3B-Base-BF16NVIDIA47.612026-08-24
72NVIDIA Nemotron 3 Nano 30B A3BNVIDIA46.812026-08-11
73Qwen3.5 35B A3BAlibaba44.212026-08-11
74Command LightCohere39.812026-07-02
75Luminous Base (13B)Aleph Alpha28.612026-07-02
76Luminous Supreme (70B)Aleph Alpha28.412026-07-02
77Luminous Extended (30B)Aleph Alpha27.212026-07-02
78OLMo (7B)AllenAI22.212026-07-02

Best tracked score per model (default configuration; source-attributed and verification-tiered). Open a model for its full benchmark surface, provenance and pricing.