1057 Performance of Grok-2, ChatGPT-4o, and Gemini 1.5 Flash Large Language Models (LLMs) in Detecting and Classifying Intracranial Hemorrhages

View at publisher Summarise with VIRAT
Authors: Albert Q. Wu, Pierce H. Davis, Avi A. Gajjar, Christopher H. Sollenberger, Zachary T. Hoglund, Saarang Patel, Kyle W. Scott, Joshua S. Catapano, Jan-Karl Burkhardt, Visish M. Srinivasan
Cited by: 0

Summary

No summary yet. VIRAT can read this paper and write a plain-language summary of its aim, methods, key findings and limitations.

Summarise with VIRAT Free to try with your Science Catalogs account.

Abstract

INTRODUCTION: Rapid interpretation of CT head scans for intracranial bleeding is critical for prioritizing neurosurgical intervention. Large language models (LLMs) have recently advanced in image analysis, with Grok-2, introduced on the social media platform X, claiming high accuracy in medical imaging interpretation. If validated, such models could serve a valuable triage function in low-resource settings and non-trauma hospitals. METHODS: Non-contrast, axial CT head scans were sourced from the RSNA 2019 database. A random sample of 400 scans (200 normal and 200 hemorrhage cases) was selected, including 40 each of epidural, intraparenchymal, intraventricular, subarachnoid, and subdural hemorrhages. Zero-shot prompting was used for each LLM to assess (1) presence of intracranial hemorrhage and (2) type of hemorrhage. A blinded medical student independently reviewed all images for comparison. McNemar’s test was applied for paired accuracy comparisons, and Cohen’s kappa assessed inter-rater agreement. RESULTS: There was no significant difference in hemorrhage detection accuracy among the LLMs, with accuracies ranging from 59.3% to 61.0%. The medical student outperformed all models in both detection accuracy and specificity. Classification performance varied, with the student significantly outperforming LLMs in detecting epidural and intraparenchymal hemorrhages. Cohen’s kappa indicated only slight agreement (κ < 0.2) between the student and each LLM, suggesting reliance on different diagnostic cues. CONCLUSIONS: Current general-purpose LLMs demonstrate moderate but inconsistent ability to detect and classify intracranial hemorrhages, underperforming compared to a human medical student. While exhibiting distinct diagnostic patterns, none of the LLMs matched human specificity or accuracy. Refinement of task-specific systems may be required to enhance clinical applicability in neuroimaging.

Fields of Study & Subjects

Artificial Intelligence in Healthcare and Education Artificial Intelligence in Healthcare Acute Ischemic Stroke Management

Publication details

Journal / source
Neurosurgery
Volume
72
Issue
Supplement_1
Pages
192–192
Publication date
26 March 2026
Language
EN