Large language model used to simulate psychiatric OSCE scenarios: a medical student perspective

Read full text Summarise with VIRAT
Authors: Zhaochu Geng, Craig S. Webster, Yan Chen, Lillian Lywan Ng, Christian U. Krägeloh, Angel Li, Marcus Alexander Henning
Cited by: 0
References: 17

Summary

No summary yet. VIRAT can read this paper and write a plain-language summary of its aim, methods, key findings and limitations.

Summarise with VIRAT Free to try with your Science Catalogs account.

Abstract

AIM: This study investigates how varying prompt conditions influence the quality and clinical coherence of responses generated by a large language model (GPT-4o mini) in simulated psychiatric OSCE scenarios. METHODS: Four psychiatric OSCE cases were presented to GPT-4o mini under four conditions with increasing details: a standard clinical prompt, a context-enhanced prompt, and two variation prompts incorporating irrelevant or distracting information. GPT-4o mini was asked to perform key OSCE tasks, history-taking, risk assessment, explanation, and management for each case. Responses were scored using a standardised, structured rubric and analysed thematically. RESULTS: GPT-4o mini generated clinically relevant responses under standard and context-enhanced prompts. However, performance declined as irrelevant information was introduced. Quantitative scores dropped significantly across the different conditions, and qualitative analysis revealed reduced coherence, increased verbosity, and difficulty prioritising clinical content. CONCLUSIONS: LLMs like GPT-4o mini can generate useful responses when provided with clear and concise prompt instructions. However, in this study, we noted that clinical accuracy and coherence deteriorated in the presence of distracting or ambiguous input. This highlights the need for critical evaluation and unambiguous literacy when using LLMs in medical education.

Fields of Study & Subjects

Artificial Intelligence in Healthcare and Education Innovations in Medical Education Artificial Intelligence in Education

Publication details

Journal / source
New Zealand Medical Student Journal
Volume
0
Publication date
28 March 2026
Language
EN