AI benchmarks

An AI benchmark is a set of tasks, a fixed way of presenting them, and a metric for scoring the responses. It is an instrument for turning a vague question — how capable is this system? — into a number that can be compared across models and over time. The best-known benchmarks are collections of exam questions, coding problems, or human preference votes 1.

Benchmarks are also incentives. When a laboratory reports a score, the next system is built to improve it, and the benchmark becomes part of the research agenda. That is useful when the benchmark captures something worth doing and harmful when it captures something easy to optimise.

This entry describes how benchmarks are constructed, the major suites and metrics, what they have made possible, the three failures that dominate the literature — contamination, saturation, and construct validity — and the dispute over whether they measure capability at all.

A benchmark has four parts: a dataset of items, a task specification, a metric, and a protocol. The dataset defines the inputs and, usually, the reference answers. The task specification says what the model is asked to do, including how many examples it sees and whether it may use tools. The metric converts outputs into a score. The protocol fixes the prompt format, sampling parameters, and reporting rules.

Most benchmarks are static: the test set is fixed, held out from training in principle, and reused for years. Static sets allow comparison over time, which is their great virtue, and they create the conditions for contamination, which is their great weakness.

Good benchmarks also publish confidence intervals and per-category results, because a single aggregate can hide the pattern. A model that is excellent at physics and poor at law can have the same average as one with the reverse profile, and the average says nothing about which is which.

Construct validity is the deeper question. A benchmark is a claim that performance on its items is evidence of the capability named in its title. That claim can be false even when the scores are accurate 2.

The modern era began with GLUE and SuperGLUE, which bundled natural-language-understanding tasks into a single leaderboard. When models saturated them, the field moved to harder and broader sets. MMLU, released in 2020, asked multiple-choice questions across dozens of academic and professional subjects and became the standard reference for general knowledge 1.

BIG-bench pushed in a different direction, assembling more than two hundred tasks contributed by a large group of researchers, including tasks designed to be difficult for models and easy for people 3. HELM responded to a different problem: a single accuracy number hid trade-offs, so it evaluated many models across many scenarios and metrics, including robustness, fairness, and cost 4.

The most consequential shift was toward tasks that resemble work. SWE-bench gives a model a real GitHub issue and a repository and asks it to produce a patch that passes the project's tests 5. Similar benchmarks now cover computer use, data analysis, medicine, and long-horizon agentic tasks. These are harder to grade, harder to keep uncontaminated, and closer to what deployment actually requires.

Alongside static sets, crowdsourced arenas rank models by pairwise human preference. They measure something different — which answer people prefer — and they avoid fixed test items, at the cost of depending on who votes and how the votes are aggregated 6.

The simplest metrics are accuracy and exact match, which suit multiple-choice questions and short answers. Classification tasks use F1, retrieval uses recall and ranking measures, and translation and summarisation use variants of overlap or model-based scoring.

Code generation introduced pass@k, the probability that at least one of k sampled programs passes the tests. It is usually estimated from a larger number of samples rather than measured directly, because generating many programs is expensive. The metric rewards exploration and can flatter a model that succeeds occasionally.

Preference evaluations use Elo ratings or Bradley–Terry models fitted to pairwise votes. These produce a ranking rather than an absolute score, and they inherit the biases of the voters: length, formatting, confidence, and agreement with the rater. Model judges introduce their own biases, including position effects and a preference for their own outputs 6.

Protocol details often matter as much as the model. The number of examples in the prompt, whether chain-of-thought is requested, the sampling temperature, and the parsing of answers can move a score by several points. A benchmark result is therefore a property of a model and a harness, not of the model alone.

Benchmarks made progress comparable. Before shared evaluation sets, a claim of improvement was difficult to assess, and results from different groups were rarely commensurable. A common task and metric let the field argue about the same thing.

They also created accountability. A laboratory that claims a capability can be asked for its scores, and independent evaluations can check them. Procurement, regulation, and journalism all rely on benchmarks because they are the only widely available quantitative evidence about systems whose internals are opaque.

For researchers, benchmarks provided a fast feedback loop. A new training method could be tested on a standard set in days, and improvements could be attributed. The resulting competition drove real gains, particularly in knowledge coverage and coding.

The cost is that the field optimises what is measured. Benchmarks became targets, and the gap between a high score and useful capability became a subject in its own right 2.

Contamination occurs when benchmark items appear in a model's training data. The model may then recall an answer rather than derive it, and the score overstates generalisation. Because training corpora are large and often scraped from the web, a public benchmark can leak in through a dataset, a forum post, or a derivative work 7.

Detection is difficult. Overlap tests compare test items with training data using n-grams, but they require access to the corpus, which closed laboratories do not provide. Perplexity and likelihood tests look for suspiciously low loss on test items; canary strings and membership inference offer other signals, none conclusive.

Contamination does not always inflate scores. A model may have seen a question but not its answer, or the memorised text may not help with the particular format. Recent work therefore distinguishes contamination of the item from contamination of the answer, and tries to measure the effect rather than the presence.

The practical response is to keep evaluation data private, generate fresh items, and test on distributions that could not have been in training. All three are expensive, and all three are in tension with the openness that makes benchmarks credible.

A benchmark saturates when leading models cluster near the ceiling and the differences between them fall within noise. MMLU, once a demanding test, is now near saturation for frontier systems. Saturated benchmarks stop discriminating, and the field replaces them with harder ones, which resets the comparison and discards the historical record.

Saturation invites teaching to the test. Models are tuned on benchmark-like data, prompts are adjusted to the benchmark's format, and training mixes are weighted toward its subjects. These are not necessarily dishonest, but they mean the score measures a narrower skill than the benchmark's name suggests.

Leaderboards add a structural problem. A crowdsourced arena depends on voluntary votes, and providers can influence the ranking by testing privately, retracting results, or releasing models selectively. A 2025 analysis documented private variants, preferential access, and unequal data contributions, and argued that these dynamics distort the ranking 8.

Goodhart's law applies: when a measure becomes a target, it ceases to be a good measure. The response is not to abandon measurement but to rotate instruments, hold some evaluations private, and report multiple metrics rather than one rank.

The strongest critique is that benchmarks measure the wrong thing. A multiple-choice exam tests recognition of correct answers, not the ability to produce them, and a coding benchmark tests short, well-specified tasks rather than the ambiguity of real software work. High scores can coexist with failures that users notice immediately 2.

Aggregate benchmarks also embed assumptions. The choice of subjects, languages, and cultural references determines who finds the test easy. A benchmark built from English-language academic material measures familiarity with that material as much as reasoning, and models trained on different corpora are compared on a scale that favours one of them.

The critique is not that benchmarks are useless but that they are under-theorised. Psychometrics has spent a century on the question of whether a test measures a construct, and AI evaluation has largely reinvented the problem without its methods. Frameworks that assess a benchmark's design, documentation, and validity are an attempt to close that gap.

A benchmark's name is a hypothesis. MMLU does not measure massive multitask understanding in the way a physicist measures mass; it measures performance on a particular set of multiple-choice questions. The distinction matters when scores are used to justify claims about capability, risk, or readiness.

One camp argues that benchmarks are broken: contaminated, saturated, gamed, and disconnected from what users care about. On this account, the numbers that dominate public discussion are closer to marketing than to measurement, and the field needs task-based, dynamic, and qualitative evaluation instead 2.

The other camp argues that benchmarks are imperfect and indispensable. They are the only instrument that permits comparison across labs, they have tracked real progress, and the alternatives — expert review, red-teaming, deployment monitoring — are expensive, subjective, and hard to scale. On this account, the remedy is better benchmarks, not fewer.

The dispute extends to policy. Governments and standards bodies lean on benchmark results when they assess risk, and companies cite them when they claim safety. Researchers point out that a benchmark score is not a risk assessment, and that a model can top a leaderboard while failing the specific evaluations that a deployment requires.

There is also disagreement about saturation. Some argue that scaling will continue to produce measurable gains on ever harder benchmarks, and that saturation is a sign of progress rather than a failure of measurement. Others argue that the remaining gaps are in capabilities that static tests cannot capture, such as reliability over long tasks, and that new instruments are needed before the next generation can be assessed at all.

The emerging consensus on evaluation is unglamorous. Keep a private held-out set, refresh it, and report confidence intervals. Publish the harness and the prompts, not only the score. Evaluate several capabilities rather than one aggregate, and include cost and latency alongside accuracy. Test on tasks that resemble the deployment, and check whether the model's failures are the ones users would notice.

For high-stakes uses, benchmarks should be only one input. Red-teaming, interpretability, incident reporting, and post-deployment monitoring provide evidence that a static test cannot. A model that scores well and behaves badly in production is a failure of evaluation design, not an anomaly.

Evaluation is becoming a field in its own right, with its own methods and its own critiques. That is a sign of maturity. The alternative — a single leaderboard treated as a verdict — is cheaper, more legible, and less true.