OpenAI Chief Scientist Jakub Pachocki published the company's internal perplexity measurements on November 22, 2026, proposing that the metric serve as a standardized basis for comparing AI model progress across labs and for setting shared safety standards2.
Perplexity, also called cross-entropy loss, measures how well a model predicts the next token in a dataset; lower values indicate stronger predictive performance. OpenAI cited a possible 2027 avalanche of AI gains and urged pacing limits tied to such metrics1.
OpenAI disclosed perplexity figures across its model lineage. GPT-2 achieved coherent text generation, GPT-3 introduced in-context learning, and GPT-3.5 enabled fluent conversational abilities with a log-perplexity of approximately 0.99. GPT-4, with a log-perplexity of approximately 0.91, demonstrated advanced chain-of-thought reasoning and opened applications in areas such as cybersecurity and mathematics.
Pachocki's post flagged three weaknesses of perplexity as a yardstick: difficulty of comparison across labs, detachment from actionable evaluations, and failure to account for advances outside of pretraining.
ANALYSIS The concession that perplexity cannot capture post-training gains — from prompting, fine-tuning, or inference-time compute — narrows the metric's usefulness precisely where recent capability jumps have concentrated.
According to Pachocki, sharing perplexity data gives competitors insights into OpenAI's progress, but the broader goal of establishing shared safety measures outweighs competitive concerns. OpenAI framed the disclosure as an effort to create a scientific foundation for pacing AI advancements responsibly.
ANALYSIS By tying a safety-pacing proposal to a metric it has already tracked internally, OpenAI positions itself to shape the terms of any future industry or regulatory framework around capability measurement. Whether rival labs adopt the same metric — given the acknowledged cross-lab comparison difficulties — remains an open question.
OpenAI's relationship with its major backer Microsoft, which is raising at least $25 billion to finance AI-driven capital expenditures[2], and its continued partnership with Nvidia Corp.[3], form the commercial backdrop against which the transparency initiative lands.