Artificial Intelligence in Cardiology Examination: A Comparative Study of Physicians, ChatGPT-4, and ChatGPT-5

Background Medicine is increasingly applying artificial intelligence (AI), particularly large language models (LLMs). Previous studies have shown that models such as GPT-4 can achieve performance comparable to physicians on medical knowledge tests. However, direct comparisons between the newest GPT-5 model, its predecessor GPT-4, and physicians are lacking, especially with respect to theoretical versus clinical question types. Methodology This comparative study analyzed 120 multiple-choice questions from the spring 2025 Polish National Cardiology Specialization Examination administered by the Medical Examination Center (CEM) in Łódź. The dataset included 62 theoretical questions and 58 clinical scenario-based questions. Examination performance of physicians who sat for the exam (n = 153) was compared with responses generated by ChatGPT-4 and ChatGPT-5. Both models were evaluated independently using the original Polish-language questions under standardized, exam-like conditions, without access to external databases or tools. Model accuracy was calculated overall and by question type. Statistical comparisons were performed using one-sample proportion Z-tests and McNemar's tests, with a Bonferroni correction applied for multiple comparisons (significance threshold set at p-values