Factors Associated with False Positive Predictions in a Logistic Regression Model of High-Intensity Mental Health Service Use

Abstract Background Predictive models in mental health can identify service users at risk of high-intensity care, enabling proactive interventions. However, false positive predictions may lead to over-medicalisation, inequitable resource use, and stigma. Understanding the factors associated with false positives can improve model interpretation and real-world application. Aims To evaluate a previously validated prediction model for high-intensity (top decile) service use and identify false positive (FP) vs. true positive (TP) predictions, examining factors across three timeframes: baseline (e.g. demographics and referral information), the prediction period (W1; the first three months after initial mental health service assessment), and the immediate (three month) period following prediction (W2). Methods Of mental health services users assessed between 2007-2024, 4,174 TP and 17,644 FP predictions were compared. Evaluated covariates included demographics, referral source, service use, diagnoses, medications, and recorded symptoms. Logistic regression was used to identify associations with FP outcomes across the three time periods. Results At baseline, FP predictions were more common among Asian service users, those living with family, and users referred by their GP, whilst TP predictions were associated with voluntary or probation service referrals. During W1, FP predictions were associated with higher community treatment days, crisis attendances, and service users managed in the outpatient setting. TP predictions were associated with antipsychotic use and engagement with multiple care teams. In W2, TP predictions were more likely among service users who remained outpatients, had higher inpatient days, or received multidisciplinary care, and FP more likely among those with substance use, multiple address changes, or ongoing crisis attendances. Conclusion The approach here to predictive modelling highlights the importance of considering features (here at baseline and W1) which may influence a model’s accuracy at the point the prediction is communicated, and those subsequent events (W2) which might indicate targets for intervention to prevent the adverse outcome. Both need to be incorporated in clinical interface communications if models are deployed in practice.