Performance of a large language model on the reasoning tasks of a physician

Science 2026

View at publisher
Authors: Peter G. Brodeur, Thomas A Buckley, Zahir Kanjee, Ethan Goh, Evelyn Ling, Priyank Jain, Stephanie Cabral, Raja-Elie Abdulnour, Adrian D. Haimovich, Jason A. Freed, Andrew Olson, Daniel J Morgan
+13 more Jason Hom, Robert Gallo, Liam G McCoy, Haadi Mombini, Christopher Lucas, M. Fotoohi, Matthew Gwiazdon, Daniele Restifo, Daniel Restrepo, Eric Horvitz, Jonathan Chen, Arjun K. Manrai, Adam Rodman
DOI: 10.1126/science.adz4433
Cited by: 47
References: 28

Abstract

More than 65 years ago, complex clinical diagnostic reasoning cases were introduced as the gold standard for the evaluation of expert medical computing systems, a standard that has held ever since. In this study, we report the results of a physician evaluation of a large language model (LLM) on challenging clinical cases across five experiments with a baseline of hundreds of physicians. We then report a real-world study comparing human expert and artificial intelligence (AI) second opinions in randomly selected patients in the emergency room of a major tertiary academic medical center. In all experiments, the LLM outperformed physician baselines and displayed continued improvement from prior generations of AI clinical decision support. Our study suggests that LLMs have eclipsed most benchmarks of clinical reasoning, motivating the urgent need for prospective trials.

Fields of Study & Subjects

Clinical Reasoning and Diagnostic Skills Artificial Intelligence in Healthcare and Education Machine Learning in Healthcare

Publication details

Journal / source
Science
Publisher
American Association for the Advancement of Science (AAAS)
Volume
392
Issue
6797
Pages
524–527
Publication date
30 April 2026
Language
EN