How an AI clinical scribe hallucinates: a source-pinned structural audit of one broadcast consultation
Abstract
A claim-by-claim structural audit of the two outputs a commercial AI clinical scribe (vendor anonymised) produced from one real consultation broadcast on public television: a clinical Summary and a patient-facing OPD letter. The source is a SHA-256-pinned verbatim transcript, so every judgement can be re-checked by hand against the exact bytes. Each substantive claim is tiered by how far it traces to the recorded source (grounded / inferred / reconstructed / fabricated / abstained). Across 40 claims the audit found 6 fabricated. The scribe was faithful to what the consultation said and failed on what it did not say: it recorded a patient's denial to a visual-disturbances question the patient was asked but never answered; it presented the nurse's later report to the doctor (a 4 a.m. headache pattern) as the patient's own history; it asserted 'No Actions for GP' from silence; and it inserted stock pleasantries as if spoken. It never once abstained. This is the characteristic failure mode of language models on partial input: smoothing an unanswered question into a recorded denial and an absence into an assertion. Framing, stated plainly: this is an ANALYTIC audit whose authority is verifiability (pinned source, line-cited, hand-checkable), NOT a deterministic-instrument output and NOT a device evaluation. The load-bearing verdicts require reading, not token overlap, which marks where deterministic checking stops and a clinician's judgement begins. One consultation, processed once; it does not claim the system is unsafe or non-compliant. The vendor is anonymised. The deterministic (In)Canon engine (frame_bind.ofm, UK patent application GB2606072.3) is a separate strand and is not part of this deposit.
Publication details
- Journal / source
- Zenodo (CERN European Organization for Nuclear Research)
- Publication date
- 4 August 2026
- Language
- EN