Inferential performance and temporal stability of large language models in suicide method prediction: A forensic psychiatric analysis

Objective: This study presents a structured evaluation of large language models (LLMs) in predicting suicide methods based exclusively on indirect forensic psychiatric indicators. Methods: Ninety-two forensic psychiatric cases (2019–2024), involving survivors of suicide attempts formally examined in medico-legal contexts, were retrospectively analyzed. Variables included age, sex, psychiatric diagnosis, previous suicide attempts, psychiatric medication use, impulsivity, and consciousness at emergency admission. Six LLMs were tested: ChatGPT-4o, ChatGPT-4o Mini, ChatGPT-O3 (OpenAI), Gemini 2.0 Flash, Gemini 2.5 Pro, and Gemini 2.5 Flash (Google DeepMind). Each case was converted into a standardized anonymized prompt. Model predictions were categorized by blinded forensic physicians and evaluated using accuracy, precision, recall, F1-score, and Cohen’s Kappa for 1-month reproducibility. Results: Gemini 2.5 Flash achieved the highest performance with 76.09% accuracy, 46.9% F1-score, and 45.2% recall. It accurately predicted the dominant method, medication overdose, but underperformed for rare categories. Temporal reproducibility was moderate (κ = 0.582), while other models exhibited lower and less stable performance. Conclusion: LLMs can infer suicide methods from indirect psychiatric data with encouraging accuracy. However, limitations in detecting rare methods and maintaining temporal consistency suggest the need for further methodological refinement and external validation prior to forensic application.