Performance Comparison of Artificial Intelligence Chatbots in Addressing Oral Lesion Queries: An In Silico Cross-Sectional Study
Summary
No summary yet. VIRAT can read this paper and write a plain-language summary of its aim, methods, key findings and limitations.
Summarise with VIRAT Free to try with your Science Catalogs account.Abstract
Background: Oral lesions are common clinical findings that frequently cause patient anxiety and prompt individuals to seek online information. Artificial intelligence (AI)-driven chatbots, such as ChatGPT, Google Gemini, and Microsoft Copilot, are increasingly utilized for immediate guidance; however, their reliability, accuracy, and safety in addressing oral lesion-related queries remain uncertain. Objective: This study aimed to evaluate and compare the performance of ChatGPT, Google Gemini, and Microsoft Copilot in responding to patient queries on oral lesions, with emphasis on accuracy, relevance, clarity, safety, transparency, and readability. Methods: Twenty patient-centered questions were curated from reputable health sources and public forums. Each question was entered into the three chatbots under standardized conditions. Four calibrated observers independently rated the responses using a structured five-point Likert scale. Readability was analyzed using the Flesch Reading Ease (FRE) and Flesch–Kincaid Grade Level (FKGL) indices. Results: Google Gemini and ChatGPT outperformed Microsoft Copilot, with significant differences observed in accuracy ( P = 0.022) and safety ( P < 0.001). Inter-rater agreement was highest for Copilot (κ ≈ 0.8), while ChatGPT demonstrated the best readability (FKGL = 6.58, FRE = 59.64). Conclusion: ChatGPT and Google Gemini demonstrated superior performance compared to Microsoft Copilot. While ChatGPT offered more readable responses, Gemini provided more comprehensive but complex content. Continuous refinement and domain-specific training are essential to enhance their clinical reliability and ensure patient safety.
Fields of Study & Subjects
Artificial Intelligence in Healthcare and Education
Artificial Intelligence in Healthcare
Health Literacy and Information Accessibility
Publication details
- Journal / source
- Journal of Indian Academy of Oral Medicine and Radiology
- Volume
- 38
- Issue
- 1
- Pages
- 82–87
- Publication date
- 1 January 2026
- Language
- EN