What AI Benchmarks Miss About Real Professional Work

Benchmarks like GDPval measure well-specified tasks with verifiable answers. They miss the ambiguous, judgment-intensive work where professional expertise still matters.

We do not know how much valuable professional work AI can competently perform. Existing benchmarks do not provide that measure.

Tame Problems

OpenAI’s GDPval benchmark represents substantial investment in evaluation. It recruits credentialed lawyers, accountants, engineers, and physicians to compare AI outputs with human work in blind evaluations of realistic tasks. It is carefully designed and transparently documented, improving on simpler metrics while measuring the wrong things for this purpose.

GDPval’s task selection imposes several limits:

Well-specified problems. Evaluators rate 89% of GDPval tasks as well-specified, with clear inputs, defined deliverables, and identifiable correct answers. Tasks judges cannot evaluate are excluded. Professional expertise often earns premium fees for ill-specified problems in which defining the question is part of the work.

One-shot evaluation. Tasks are completed in a single session, averaging around seven hours, with no iteration, no clarification requests, no back-and-forth. Real professional work involves extended engagement: discovering hidden constraints, refining approaches based on feedback, asking the follow-up questions that distinguish expert from competent.

Bounded time horizons. Seven hours is long for a benchmark but short for professional judgment. It cannot include the weeks or months of context a physician gains managing a chronic condition, a lawyer handling a transaction, or an engineer monitoring construction.

Verifiable answers. GDPval requires assessable outputs. Even among the tasks simple enough for inclusion, inter-rater reliability is only 71%.

OpenAI acknowledges these limitations. Their documentation notes that GDPval tests “discrete deliverables” rather than “extended professional relationships,” and explicitly excludes “iterative refinement” and “ambiguous requirements.”

These documented exclusions mean GDPval measures a different kind of work from the professional judgment that commands premium fees.

Wicked Problems

The distinction I’m drawing comes from planning theory: “tame” versus “wicked” problems.

Tame problems have clear formulations, definable solutions, and objective success criteria. They can be technically difficult, but evaluators can establish whether a solution succeeds. Most benchmarks test tame problems because those are the problems benchmarks can test.

Wicked problems resist formulation. The problem definition depends on the approach you take. Stakeholders disagree about what success means. Solutions aren’t right or wrong but better or worse, and “better” is contested. Every attempt to solve the problem changes the problem.

Professional expertise commands its highest premiums on wicked problems. The contested diagnosis where reasonable physicians disagree. The transaction where legal strategy depends on predictions about counterparty behavior. The engineering judgment that weighs incommensurable values under uncertainty.

For this class of problem, we have almost no systematic measurement of AI capability. Not because measurement is impossible, but because the methodologies that work for tame problems break down.

If three senior physicians disagree and AI agrees with one, there is no agreed diagnosis against which to score the model. The difficulty concerns what capability means in this case, as well as how to measure it.

The Ground-Truth Problem

Benchmarks require ground truth: a correct answer to measure against. For tame problems, ground truth exists. For wicked problems, it often doesn’t. The problems that define high-end professional work are precisely the problems where “correct” is contested.

We can still measure things:

• Whether AI recommendations fall within the range of expert opinion
• Whether AI-assisted outcomes differ systematically from human-only outcomes
• Whether AI identifies considerations experts overlooked
• Whether AI reasoning is coherent, defensible, well-supported
• Whether AI appropriately flags uncertainty and limits

None of these is “what percentage of professional work can AI do?” Professional judgment on hard problems doesn’t reduce to a percentage. But these measures would reveal whether AI is becoming a credible participant in professional judgment, even without a single scalar metric.

The infrastructure to generate this evidence mostly doesn’t exist. What exists instead are benchmarks on tame problems, extrapolated to claims about wicked ones.

The Wrong Frame

Current benchmarks also evaluate AI as an oracle, comparing its answer quality with that of human professionals.

But the oracle framing is itself problematic. When you ask AI for answers, you inherit the verification problem: you can’t confirm the answer is right without the expertise to produce it yourself. Benchmark evaluators have that expertise. End users often don’t. The benchmark measures something real for the evaluators; it tells users almost nothing about whether they can safely rely on the output.

If professional work should use expert-specified execution with inspectable methodology, answer-quality benchmarks do not assess the required process. No benchmark asks whether that process is defensible, inspectable, or methodologically sound.

The measurement gap isn’t just tame versus wicked. It’s answer versus process. The infrastructure is built around an approach that may not be the right one for consequential work.

Who Builds Measurement

The parties able to fund evaluation have conflicting incentives about what to measure.

AI vendors invest heavily in evaluation: by some estimates, 5000:1 relative to government and academic investment combined. They have the models, the technical capability, and the budgets. GDPval, APEX, and similar efforts represent genuine methodological work.

But vendor-funded evaluation faces unavoidable incentive problems:

• Labs routinely produce dozens of model variants, publishing benchmarks only for highest-scoring versions
• Benchmarks showing unfavorable results get deprecated; favorable ones proliferate
• Training on benchmark data (teaching to the test) is endemic
• Mathematical reasoning benchmarks have been repeatedly compromised by data contamination

Investors, customers, and the press assess vendors by benchmark performance. Vendors therefore have an incentive to optimize those metrics even when they poorly reflect practical capability.

Academic researchers, professional bodies, and regulators have credibility to evaluate independently but lack resources. Government spending is negligible compared with industry spending. Academic funding rewards publication rather than practical evaluation, while professional bodies work slowly and face internal politics.

Parties with a stake in particular results control the measurement infrastructure. Independent evaluation receives far less support than it needs.

Who Benefits from the Gap

Several groups benefit from missing measurements.

AI vendors benefit from ambiguity when capability is uncertain. They can market possibility rather than demonstrated performance. When capability is high, they benefit from measurement: rigorous evaluation becomes marketing. Either way, they control what gets measured and what gets published.

Professional services firms benefit from unfalsifiable claims. As long as AI capability remains uncertain, “AI can’t do what we do” is a marketing position that can’t be tested. The measurement vacuum protects premium pricing.

Clients and regulators lose. They need accurate capability information to make decisions. They can’t produce it themselves. They’re dependent on parties with interests in particular answers.

The parties who would benefit most from accurate, independent measurement are precisely the parties without resources or expertise to produce it.

The Timing Problem

Professional liability insurers face a particular timing problem.

Insurers price professional liability in engineering, medicine, law, and accounting using decades of claims data, underwriting questionnaires, loss-control requirements, and actuarial expertise.

Those mechanisms do not yet measure current AI use.

Professional liability claims take three to seven years from incident to resolution. Current claims data reflects practice from 2018–2020, preceding major changes in AI capability and adoption. Claims experience will not reflect today’s AI use for years.

Insurers have the incentives and resources to assess AI risk, but need measurements that work faster than the claims cycle. AI capability and adoption are changing faster than their traditional data can record.

I have seen no evidence that insurers have solved this mismatch. Their existing mechanisms were not designed for capability changing at this rate.

Evaluation Methods and Funding

Better measurement would look different from current benchmarks:

Test wicked problems. Develop evaluations with ambiguous situations, contested answers, extended time horizons, and iterative engagement. These are harder to design and more relevant to professional judgment.

Independent funding. Shift the 5000:1 ratio. Academic grants, professional body investment, regulatory funding, philanthropic support for evaluation infrastructure that isn’t controlled by those with stakes in outcomes.

Practitioner red-teaming. Working professionals should try to make AI fail on cases from their own work, extending evaluation beyond problems benchmark designers devise.

Outcome tracking. Connect AI use to downstream results: patient outcomes, case results, project success. The infrastructure to do this mostly doesn’t exist. Building it would reveal what capability actually means in practice.

Calibration measurement. Does AI know what it doesn’t know? Current evidence suggests models confidently generate plausible nonsense on topics outside their training. Measuring whether stated confidence matches actual accuracy is tractable and critical.

These evaluations are feasible but receive insufficient investment under current incentives.

Decisions Without Adequate Evidence

Existing benchmarks leave AI’s capability on ambiguous professional work unresolved.

GDPval and similar efforts measure something real. That something is performance on tame problems: well-specified tasks with verifiable answers and bounded time horizons. These benchmarks show AI performing competently, sometimes impressively, on this class of problem.

Whether that performance extends to wicked problems (the ambiguous, contested, judgment-intensive work that defines senior professional expertise) is a different question. Current benchmarks don’t answer it. Current measurement infrastructure can’t answer it.

The gap matters because decisions are being made on both sides. Firms are adopting AI without systematic evaluation of whether it works on their actual problems. Clients are paying premiums for human judgment without knowing whether that judgment adds value. Insurers are pricing risk based on assumptions that may not hold. Regulators are deferring oversight because they lack capability data.

These decisions depend on uncertain assumptions, some of which will be wrong. Better measurement would reduce that uncertainty but requires investment that current incentives do not produce.

I do not have a solution beyond identifying the problems: existing benchmarks do not measure what matters, those who could improve measurement have conflicting incentives, and those who need the information cannot produce it themselves.

The measurement gap follows from how evaluation is funded and what it tests. It will persist without changes to those arrangements.