Protocol deviation outlier estimation combined with generative AI

OBJECTIVE: Protocol deviations (PDs) are critical indicators of data quality and operational risk in clinical trials. Systematic under- or over-reporting of PDs at investigator sites can obscure true trial performance and compromise regulatory compliance. The goal of this paper is to present an integrated approach combining statistical outlier detection with generative AI-powered content analysis for detecting and characterizing outlier sites in PD reporting, enabling proactive risk management and targeted site oversight. RESULTS: In this study, the simaerep bootstrap algorithm was applied across 578 clinical studies encompassing 39,936 sites. It is shown that the method successfully identifies sites exhibiting statistically significant deviations from expected PD reporting rates, flagging both potential under-reporters and high-rate reporters. In addition, generative AI combined with hierarchical clustering automatically categorized diverse PD narratives into a validated taxonomy of 15 major categories and 47 specific topics through a "human-in-the-loop" refinement process with clinical operations experts. Independent validation of the GenAI topic classification using an "LLM-as-judge" approach achieved 88.7% accuracy (95% CI: 86.5%–90.6%). This integrated capability equips clinical operations teams to identify sites warranting further investigation and determine whether observed deviation patterns are study-systemic, country-specific, or site-specific, ultimately improving data integrity and risk-based audit strategy.