Expert evaluation of GPT-4o and Gemini responses to patient questions on carotid endarterectomy
Abstract
OBJECTIVE: The aim of this study was to compare the accuracy, scientific quality, and clarity of responses generated by GPT-4o and Gemini to frequently asked patient questions related to carotid artery disease and carotid endarterectomy. METHODS: In total, 40 unique carotid endarterectomy-related questions were compiled from online sources and clinical experience. Each was entered into separate new sessions with GPT-4o and Gemini 2.5 Flash in Turkish, and responses were collected without modification. Notably, four blinded cardiovascular surgeons independently rated each answer (1-5 Likert scale) in three domains: Accuracy, Scientific Quality, and Clarity. Mean response lengths and domain scores were compared using appropriate paired tests. RESULTS: GPT-4o produced longer responses than Gemini (258.1±101.6 vs. 193.2±43.7 words; p
Fields of Study & Subjects
Artificial Intelligence in Healthcare and Education
Congenital Heart Disease Studies
Reliability and Agreement in Measurement
Publication details
- Journal / source
- Revista da Associação Médica Brasileira
- Publisher
- FapUNIFESP (SciELO)
- Volume
- 72
- Issue
- 2
- Pages
- e20251453–e20251453
- Publication date
- 1 January 2026
- Language
- EN