Every number traces back.
Ten ratified manifestos define what counts: 64 benchmarks in the baskets, 0 retired for saturation or contamination. Every score wears one of five provenance marks — and below, today's reasoning leader is followed from source to rank.
A third party ran the model.
Replicable from the harness's published prompts and scoring code. The strongest mark a score can wear.
An aggregator surfaced the score.
Model card, leaderboard table, registry entry — a neutral party re-attesting a primary source.
The maker's own number.
System card, model card, launch post. Authoritative on the model — not independently replicated.
A vendor re-states a rival's number.
Launch-post comparison tables, a blog quoting a competitor's eval. Kept, attributed — and never the headline.
Extracted from prose, not re-run.
Research papers, conference talks, technical briefs. Surfaced with its source — the lowest rung, worn openly.
Ingest
200+ sources poll on schedule — leaderboards, system cards, audited harnesses, papers.
Canonicalize
Aliases collapse onto one canonical model and one canonical benchmark; most-recent wins at the database layer.
Score
Per-axis manifestos define the basket. Composites are reliability-weighted percentiles, shrunk toward the median for thin samples.
Verify
Every score carries its provenance mark; within-model disagreement surfaces as a band, never averaged away.
Publish
The boards render straight from the scored view — no editorial overlay between the data and the page.
Reasoning
Tier 1 · 2026-Q3Ranked on critpt, HLE, ARC-AGI-2/3, GPQA Diamond, SimpleBench and LiveBench reasoning (2026-Q3 v2, ratified 2026-08-23). Weights follow measured frontier separation, construct centrality and field recognition; no saturated benchmark ranks.
Coding
Tier 1 · 2026-Q3Ranked on the SWE-bench family, LiveBench agentic coding + coding, Terminal-Bench Hard + 2.1, scicode, LiveCodeBench v6 and the webdev arena (2026-Q3 v3, AA Coding Index removed 2026-09-19 after AA withdrew it; replaced 1:1 by Terminal-Bench Hard).
Agentic
Tier 1 · 2026-Q3Ranked on TauBench V3, OSWorld-Verified, MCP Atlas, GDPval, Terminal-Bench 4.0, BrowseComp and analystAgent (2026-Q3 v4). apexAgents/itbenchSre moved to DEPTH — too thin (33 models) to rank the newest launches; Terminal-Bench 4.0 (115 models incl. GPT-6 Astra, 27 vendors) added as the broad agentic-execution leg — operator Option A, 2026-09-20.
Safety
Tier 1 · 2026-Q3Scale's SEAL safety boards (MASK honesty, Fortress adversarial risk, PropensityBench misuse propensity) now flow daily — 46 models measured on two or more. The cross-vendor safety ranking is parked pending calibration research; board scores appear on each model's page, labeled, with direction stated (Fortress and PropensityBench count risk — lower is better).
Long Context
Tier 2 · 2026-Q3Ranked on MRCR v2, AA-LCR and LongBench v2 (2026-Q3 v2.1, coverage pass 2026-08-23). No living universal long-context board exists beyond AA's; card depth carries the rest.
Math
Tier 2 · 2026-Q3Ranked on FrontierMath Tier 4 and Tiers 1-3 v2, MathArena BrokenArXiv and ArxivMath (newest live monthly cut) and LiveBench Mathematics (2026-Q3 v2.7, 2026-09-10). Every ranked entry currently scores newly released models. MathArena's cross-month "overall" boards are tracked as depth rather than ranked: they only score a model that has been run on every live monthly cut, and print N/A for four current flagships including GPT-5.6 Sol and Claude Opus 5. The competition boards — AIME 2026, HMMT Feb 2026, IMO-AnswerBench, Apex 2025, USAMO 2026 — are tracked as depth: they are no longer re-run on new launches, and AIME is saturated at the frontier (every measured top-25 model scores 95+). Depth entries stay visible on model pages and re-promote if a board resumes.
Multimodal
Tier 2 · 2026-Q3Ranked on MMMU-Pro, OCRBench, Video-MME, MMMU, CharXiv (reasoning) and MathVista (2026-Q3 v2.3, ratified 2026-09-08). MMMU and MMMU-Pro are ONE suite — they agree on 97.6% of ranked pairs, so the pair is capped at 0.30 of the axis rather than counted twice. OCRBench is admitted as the most independent entry measured (text-in-image). BLINK, MMStar, OCRBench v2 and the vision arena stay tracked as depth, visible on model pages but not ranked.
Multilingual
Tier 2 · 2026-Q3Scale's MultiNRC multilingual board now flows (36 models, 8 vendors — French, Spanish and Chinese). A single-board three-language ranking is parked pending research; multilingual scores appear on model pages. Re-opens with a broader source or the research pass.
Factuality
Tier 2 · 2026-Q3Ranked on AA-Omniscience (accuracy + non-hallucination), SimpleQA Verified and Vectara HHEM (2026-Q3 v2, ratified 2026-08-23). Third-party day-0 coverage is AA's alone; new launches reach eligibility as Epoch and Vectara publish.
Instruction Following
Tier 3 · 2026-Q3Ranked on LiveBench instruction following, MultiChallenge, IFBench and Multi-IF (2026-Q3 v2.1, coverage pass 2026-08-23).
785 models carry a permalink on Vector Wire, which is 307,720 possible pairings. Our sitemap submits 1,770 of them — 0.58% — the 60 most-benchmarked models crossed pairwise, and nothing else.
- ComparisonsSubmitted only for the 60 models carrying the most tracked benchmarks, crossed pairwise. A submitted comparison is withheld from indexing when the two models share fewer than 10 comparable benchmarks — below that there is not enough overlap to compare, and we would rather publish nothing than pad a table.
- Benchmark boardsA leaderboard is published once at least 5 published models carry a comparable default-variant score on it. Internal and scaffold evaluations are excluded outright. Every board we publish is listed here.
- Model pagesOne page per published model. No variants, no per-benchmark permutations, no per-vendor duplicates of the same model.
That ratio describes the URLs we submit for indexing. The comparison route itself answers any two tracked models, and a model page links the comparisons most relevant to that model whether or not the pair is submitted — a reader following a link is never sent to a dead end. What is deliberately bounded is what we ask a search engine to index.