Skip to content
VECTOR WIREAI INTELLIGENCE
⌘K
UTC
Models741Deals30d261Policy30d65, 14 pendingSources1,013

vectara_answer_rate leaderboard — the family board.

4 TRACKED VARIANTS · 2 FIRST-CLASS BOARDS

One family, 4 tracked prints: vectara_factual_consistency is cited standalone and keeps live board of its own. Scoring configs and splits of the base board render as sections of this page, each under its own honest label.

● LIVE DATA102 MODELS ON THIS BOARDLEADER MERCURY 2 AT 100

First-class boards in this familyCITED STANDALONE · EACH KEEPS ITS OWN LIVE BOARD

#BoardModels scored
01vectara_answer_rateTHIS PAGE102
02vectara_factual_consistency102

The board — vectara_answer_rateTOP 100 OF 102 MEASURED · BEST SCORE PER MODEL, DEFAULT VARIANT

#ModelVendorScoreVLast seen
01Mercury 2Inception1002026-05-02
02GPT-5.4 nanoOpenAI1002026-05-02
03GPT-5.1OpenAI1002026-08-29
04GPT-6 SolOpenAI1002026-09-23
05o3-proOpenAI1002026-08-29
06Nova MicroAmazon1002026-05-02
07GPT-5.4 miniOpenAI1002026-05-02
08GPT-5.2OpenAI1002026-06-22
09GPT-5 nanoOpenAI1002026-06-18
10Gemma 4 31BGoogle1002026-05-02
11Granite 3.3 8B InstructIBM1002026-05-02
12GPT-5.4 ProOpenAI1002026-05-02
13GPT-5.5OpenAI1002026-05-02
14Granite 4.0 H SmallIBM1002026-06-19
15Llama 4 Maverick 17B 128E Instruct FP8Meta1002026-05-02
16GPT-6 AstraOpenAI1002026-09-23
17Ministral 8B 2410Mistral99.92026-05-02
18Qwen3 32BAlibaba99.92026-05-02
19GPT-5OpenAI99.92026-08-10
20GPT-5.4OpenAI99.92026-05-02
21Mistral Large 2.1Mistral99.92026-05-02
22GPT-4.1OpenAI99.92026-05-02
23Qwen3 14BAlibaba99.92026-05-02
24Ministral 3B 2410Mistral99.92026-05-02
25Nova LiteAmazon99.92026-05-02
26GPT-5 miniOpenAI99.92026-05-02
27Claude Sonnet 4.6Anthropic99.92026-05-02
28Qwen3 4BAlibaba99.92026-05-02
29gpt-oss-120bOpenAI99.92026-05-02
30Qwen3 8BAlibaba99.92026-05-02
31Qwen3.5 FlashAlibaba99.82026-05-02
32Qwen3.5 35B A3BAlibaba99.82026-05-04
33Gemma 4 26B A4BGoogle99.82026-05-02
34Claude Opus 4.6Anthropic99.82026-05-02
35Qwen3.5 122B A10BAlibaba99.82026-08-29
36GLM-4.7Z.ai99.82026-05-02
37Aya Expanse 32BCohere99.82026-05-02
38Qwen3.5 Plus 02.15Alibaba99.82026-05-02
39Qwen3.5 27BAlibaba99.82026-05-02
40Gemini 3 Flash PreviewGoogle99.82026-05-02
41GLM-5Z.ai99.72026-05-02
42Grok 4.1 FastSpaceXAI99.72026-08-29
43Mistral Medium 3.1 (Non-Reasoning)Mistral99.72026-06-23
44Kimi K2.6Moonshot99.72026-05-12
45Gemini 3.1 Flash Lite PreviewGoogle99.62026-05-02
46Ministral 3 14BMistral99.62026-05-02
47NVIDIA Nemotron 3 Nano 30B A3BNVIDIA99.62026-08-29
48Jamba Mini 2AI2199.62026-05-02
49Nova 2.0 LiteAmazon99.62026-05-02
50finix_s1_32bantgroup99.52026-05-02
51Gemini 2.5 Flash LiteGoogle99.52026-05-02
52Grok 4 FastSpaceXAI99.52026-08-29
53Llama 3.3 70B InstructMeta99.52026-05-02
54Claude Haiku 4.5Anthropic99.52026-05-02
55Gemini 3 ProGoogle99.42026-05-02
56Gemini 3.1 ProGoogle99.42026-05-02
57MiniMax M2.7MiniMax99.42026-05-02
58Nova ProAmazon99.32026-05-02
59o4-miniOpenAI99.22026-08-29
60Ministral 3 8BMistral99.12026-05-02
61GPT-5.6 SolOpenAI99.12026-09-23
62Jamba 1.7 MiniAI2199.12026-05-02
63Gemini 2.5 ProGoogle99.12026-05-02
64Trinity Large ThinkingArcee AI992026-05-02
65Gemini 2.5 FlashGoogle992026-05-02
66Llama 4 ScoutMeta992026-05-02
67Jamba 1.7 LargeAI2198.92026-05-02
68Gemma 3 27BGoogle98.82026-05-02
69Mistral Large 3Mistral98.82026-05-02
70Claude Opus 4.5Anthropic98.72026-06-20
71Claude Sonnet 4Anthropic98.62026-06-20
72Kimi K2 (Non-Reasoning)Moonshot98.62026-06-20
73Kimi K2 InstructMoonshot98.62026-05-02
74MiniMax M2.1MiniMax98.52026-05-02
75MiniMax M2.5MiniMax98.22026-05-02
76GLM-4.5-AirZ.ai98.12026-05-02
77Claude Opus 4.7Anthropic982026-08-29
78Mistral Small 3 (Non-Reasoning)Mistral97.92026-05-02
79Command ACohere97.62026-05-02
80DeepSeek-V3DeepSeek97.52026-05-02
81Gemma 3 12BGoogle97.42026-05-02
82DeepSeek-V4-ProDeepSeek97.22026-06-29
83DeepSeek-R1DeepSeek972026-08-29
84DeepSeek-V3.2-ExpDeepSeek96.62026-05-02
85Claude Sonnet 4.5Anthropic95.62026-05-30
86Command R+ (Apr '24)Cohere952026-05-02
87Qwen3 235B A22BAlibaba94.92026-05-02
88DeepSeek-V3.1DeepSeek94.52026-05-02
89GLM-4.6Z.ai94.52026-05-02
90Qwen3 Next 80B A3BAlibaba94.42026-08-29
91GPT-4oOpenAI93.82026-05-02
92Grok 3SpaceXAI932026-06-19
93DeepSeek-V3.2DeepSeek92.62026-05-02
94Phi 4 Mini InstructMicrosoft92.52026-06-19
95Claude Opus 4.1Anthropic92.42026-06-18
96Kimi K2.5Moonshot92.22026-05-02
97GLM-4.7-FlashZ.ai91.62026-05-02
98Claude Opus 4Anthropic912026-06-20
99Phi 4Microsoft80.72026-05-02
100Aya Expanse 8BCohere77.52026-05-02
◆ IV — INDEPENDENTLY VERIFIED · ◆ AA — AGGREGATOR · ◆ VA — VENDOR · ◇ SA — SOURCE-ATTRIBUTED

Variants of this boardSCORING CONFIGS & SPLITS — NOT CITED STANDALONE, SO THEY LIVE HERE

vectara_avg_summary_length

VARIANT102 models · best score per model
#ModelVendorScoreVLast seen
01Phi 4 Mini InstructMicrosoft420.22026-06-19
02GPT-5.1OpenAI254.42026-08-29
03Ministral 8B 2410Mistral1962026-05-02
04GPT-5.2OpenAI186.32026-06-22
05Grok 4 FastSpaceXAI173.92026-08-29
06finix_s1_32bantgroup172.42026-05-02
07GPT-5 miniOpenAI169.72026-05-02
08Ministral 3B 2410Mistral167.92026-05-02
09GPT-5OpenAI162.72026-08-10
10DeepSeek-V4-ProDeepSeek153.82026-06-29
11Claude Opus 4.7Anthropic149.12026-08-29
12Mercury 2Inception149.12026-05-02
13GPT-6 AstraOpenAI148.52026-09-23
14GPT-5.4 ProOpenAI148.52026-05-02
15Claude Sonnet 4Anthropic145.82026-06-20

vectara_hallucination_rate

VARIANT102 models · best score per model · lower is better
#ModelVendorScoreVLast seen
01finix_s1_32bantgroup1.82026-05-02
02GPT-5.4 nanoOpenAI3.12026-05-02
03Gemini 2.5 Flash LiteGoogle3.32026-05-02
04Phi 4Microsoft3.72026-05-02
05Llama 3.3 70B InstructMeta4.12026-05-02
06Arctic InstructSnowflake4.32026-05-02
07Gemma 3 12BGoogle4.42026-05-02
08Mistral Large 2.1Mistral4.52026-05-02
09Qwen3 8BAlibaba4.82026-05-02
10Nova 2.0 LiteAmazon5.12026-05-02
11Nova ProAmazon5.12026-05-02
12Mistral Small 3 (Non-Reasoning)Mistral5.12026-05-02
13Granite 4.0 H SmallIBM5.22026-06-19
14Gemma 4 26B A4BGoogle5.22026-05-02
15Jamba Mini 2AI215.32026-05-02
THE FAMILY BOARD · EVERY PRINT LABELED · EVERY SCORE SOURCE-ATTRIBUTED ON ITS OWN LIVE BOARDAll leaderboardsThe frontier boardModels by vendor