Performance Comparison of Artificial Intelligence Chatbots in Addressing Oral Lesion Queries: An In Silico Cross-Sectional Study

Read full text Summarise with VIRAT
Authors: V Balajee, Roland Prethipa, Ayesh Das, Sirisha Emani
Cited by: 0
References: 19

Summary

No summary yet. VIRAT can read this paper and write a plain-language summary of its aim, methods, key findings and limitations.

Summarise with VIRAT Free to try with your Science Catalogs account.

Abstract

Background: Oral lesions are common clinical findings that frequently cause patient anxiety and prompt individuals to seek online information. Artificial intelligence (AI)-driven chatbots, such as ChatGPT, Google Gemini, and Microsoft Copilot, are increasingly utilized for immediate guidance; however, their reliability, accuracy, and safety in addressing oral lesion-related queries remain uncertain. Objective: This study aimed to evaluate and compare the performance of ChatGPT, Google Gemini, and Microsoft Copilot in responding to patient queries on oral lesions, with emphasis on accuracy, relevance, clarity, safety, transparency, and readability. Methods: Twenty patient-centered questions were curated from reputable health sources and public forums. Each question was entered into the three chatbots under standardized conditions. Four calibrated observers independently rated the responses using a structured five-point Likert scale. Readability was analyzed using the Flesch Reading Ease (FRE) and Flesch–Kincaid Grade Level (FKGL) indices. Results: Google Gemini and ChatGPT outperformed Microsoft Copilot, with significant differences observed in accuracy ( P = 0.022) and safety ( P < 0.001). Inter-rater agreement was highest for Copilot (κ ≈ 0.8), while ChatGPT demonstrated the best readability (FKGL = 6.58, FRE = 59.64). Conclusion: ChatGPT and Google Gemini demonstrated superior performance compared to Microsoft Copilot. While ChatGPT offered more readable responses, Gemini provided more comprehensive but complex content. Continuous refinement and domain-specific training are essential to enhance their clinical reliability and ensure patient safety.

Fields of Study & Subjects

Artificial Intelligence in Healthcare and Education Artificial Intelligence in Healthcare Health Literacy and Information Accessibility

Publication details

Journal / source
Journal of Indian Academy of Oral Medicine and Radiology
Volume
38
Issue
1
Pages
82–87
Publication date
1 January 2026
Language
EN