Quantitative Synthesis of Large Language Model Performance in Medical Reasoning Tasks

View at publisher Summarise with VIRAT
Authors: Pavan D. Paikrao, Meenal Wagh, Sayali S. Chavan, Sukumar Chougule, Rasika Vishal Wattamwar, Farhina Anjum Sayyad, D. G. Bhalke, Chaitali Deshpande
Cited by: 0
References: 20

Summary

No summary yet. VIRAT can read this paper and write a plain-language summary of its aim, methods, key findings and limitations.

Summarise with VIRAT Free to try with your Science Catalogs account.

Abstract

This study presents a comprehensive evaluation of AI diagnostic models across diverse clinical cases sourced from Thieme (77 cases) and Elsevier (48 cases). The dataset spans frequent (42 cases), less frequent (44 cases), and rare (39 cases) conditions, ensuring balanced assessment. Diagnostic performance, treatment recommendation accuracy, and linguistic reliability were compared across multiple state-of-the-art AI systems. Results highlight strong overall diagnostic capabilities, with notable variations across specialties and disease frequencies. While Pediatrics consistently demonstrated the highest performance, Surgery emerged as the most challenging specialty. Among models, GPT-4o achieved superior diagnostic consensus, treatment recommendation accuracy, and linguistic precision, underscoring its clinical utility. The findings provide empirical benchmarks for advancing AI-based medical decision support systems.

Fields of Study & Subjects

Artificial Intelligence in Healthcare and Education Artificial Intelligence in Healthcare Machine Learning in Healthcare

Publication details

Journal / source
Biennial International Conference on Nascent Technologies in Engineering (ICNTE)
Pages
1–4
Publication date
16 January 2026
Language
EN