Atomic Fact-Checking Increases Clinician Trust in Large Language Model Recommendations for Oncology Decision Support: A Randomized Controlled Trial

Abstract Background Large language models (LLMs) can synthesize clinical guidelines and generate diagnostic and treatment recommendations, yet clinician trust remains a barrier to adoption. Traditional approaches emphasize natural language explanations of LLM-aided recommendations and source citations, but their effectiveness in high-stakes clinical settings is uncertain. Objective To determine whether atomic fact-checking (AFC), which decomposes AI recommendations into individually verifiable claims linked to source guideline documents, increases clinician trust compared to explanations, citations, and other traditional transparency interventions. Methods A randomized controlled trial was conducted with 356 clinicians (160 radiologists, 85 radiation oncologists, 111 medical oncologists). Participants evaluated AI-generated recommendations for 21 oncology cases across seven cancer types (prostate, breast, lung, colorectal, kidney, liver, lymphoma), yielding 7,476 trust ratings. Participants were randomly assigned to one of two arms (with or without natural language explanations) and further sub-randomized to one of five transparency conditions: (1) recommendation only, (2) recommendation with explanation, (3) recommendation with source citation, (4) recommendation with explanation and citation, or (5) recommendation with explanation, citation, and AFC. The primary outcome was trust measured on a validated 5-point Likert scale. Results AFC produced significantly higher trust than all control conditions. Mean trust scores were 2.59 (95% CI, 2.54–2.64) for recommendation only, 2.84 (2.79–2.90) for explanation, 3.01 (2.97–3.05) for source citation, 3.09 (3.04–3.14) for combined explanation and citation, and 3.80 (3.76–3.84) for AFC. The effect size for AFC versus pooled controls was Cohen’s d = 0.94 (95% CI, 0.88-1.00; P < .001). The proportion expressing trust (score ≥ 4) increased from 26.9% to 66.5% with AFC (absolute increase: 39.5 percentage points; number needed to treat: 2.53). Effects were consistent across specialties (d = 0.80–1.03), cancer types (d = 0.79–1.10), and experience levels (d = 0.62–1.41). Conclusions AFC substantially increases clinician trust in AI-generated oncology recommendations. Decomposing AI outputs into verifiable claims with linked guideline sources produces larger effects than traditional transparency mechanisms.