How HealthBench Professional works
OpenAI · 525 tasks · updated August 17, 2026
HealthBench Professional is OpenAI's April 2026 successor to HealthBench. It holds 525 tasks that physicians selected out of 15,079 real conversations clinicians had with AI at work, spanning care consults, clinical documentation, and medical research. Each task carries a rubric written for it, and responses score from 0 to 1. It exists because the parent set had stopped separating strong models from one another.
| publisher | OpenAI |
|---|---|
| released | 2026-04 |
| size | 525 physician-authored tasks |
| scale | 0 to 1, higher is better |
How the tasks were built
Physicians read 15,079 real clinician conversations and kept 525 of them. Selection favored hard cases: difficulty in the kept set is enriched about 3.5 times over the source pool, and roughly one third of the tasks are adversarial. 190 physicians from 50 countries and 26 specialties worked on the set.
Each task then received its own rubric, a list of criteria an answer should meet with points attached to each. Rubrics went through three stages of adjudication, with three or more physicians reviewing every criterion. The rubric, not an answer key, is what the benchmark stores.
How responses are graded
A model does the grading. The default grader is GPT-5.4 at low reasoning effort. It reads the task, the response, and the rubric, then marks each criterion met or unmet. The score is points earned divided by the maximum positive points available, so it measures rubric credit rather than accuracy.
A length adjustment then applies. Long answers collect rubric points by covering ground at random, so verbosity is penalized at about 1.47 points per 500 characters beyond 2,000. Published numbers are usually length-adjusted, so check which variant a table shows before you compare rows.
What a score means
Physician-written responses score 0.437 on these same rubrics. Use that as your anchor. A 0.5 is not half correct; it is above what the physicians who built the set produced when they answered the tasks themselves, because the rubrics reward completeness and context checks that people writing prose tend to skip. Comparability depends on the grader model, its reasoning effort, and the length adjustment all matching across rows.
Limits
The grader is a model, so it carries its own error rate, and that error repeats across every row rather than averaging out. Rubrics capture what the authoring physicians thought to write down, and nothing else. Because tasks were chosen for difficulty and about a third are adversarial, the mix does not resemble the questions a deployed system actually receives, so a score is not an estimate of field performance. The source conversations came from clinicians using AI at work, which makes this a measurement of professional use rather than patient-facing use.
Current standings are kept on healthbenchprofessional.com, with cross-benchmark context at clinicalbenchmarks.ai.
The grading machinery behind pages like this one is covered in the rubric grading and model graders guides.