CRLN-HB-001 — The Human Baseline: a benchmark for clinical research AI, scored against people

Read full text
Authors: Joshua Webber
Cited by: 2

Abstract

Clinical research is being rapidly automated, yet no published human baseline exists against which AI systems for protocol authoring, deviation triage, monitoring or safety coding can be compared. Vendors report accuracy figures with no denominator. This report establishes one. It reports the distribution of 942 rubric-scored decisions made by 57 clinical research professionals across 16 countries on standardised Good Clinical Practice scenarios, mapped to CRLN-TRG-001 and through it to the Joint Task Force Harmonized Core Competency Framework. The baseline is severely ceiling-compressed and the report leads with that finding rather than burying it: 747 of 942 decisions (79.3%) scored 100, and the median human score is 100 in every competency domain. A model scoring 91.4 is therefore below the median human, not equal to it, and the benchmark has little power to separate an excellent model from a merely good one. All discriminating signal lies in the 20.7% of decisions where humans do not score perfectly. The specification defines a required reporting format (overall, per-domain, tail-restricted, and a disagreement profile listing items where a model chose a non-compliant action), a development and held-out split for contamination control, and the limitations and conflicts of interest that constrain the instrument. Version 0.2 records an instrument change: twelve harder items were authored for the most ceiling-bound domain and merged on 10 September 2026. Those items carry zero responses, so the baseline reported here has not moved, and the release says so explicitly rather than leaving a reader to infer it.

Publication details

Journal / source
Zenodo (CERN European Organization for Nuclear Research)
Publication date
10 September 2026
Language
EN