Predicting PANSS symptoms in schizophrenia spectrum disorders using speech only: an international, multi-centre, retrospective, computational study across multiple languages

Summary Background speech carries cues to variation in mental state in schizophrenia spectrum disorders/psychotic disorders, typically indexed with clinician-rated scales such as the PANSS. Progress in the automation of speech-based symptom modelling has been constrained by data scale and the underrepresentation of low-resource languages. In this study, we aggregate multi-center recordings to assemble a large corpus and assess symptom-prediction models at scale, to enable more objective and efficient assessments and the early detection of relapse-related signals from speech. Methods We compiled data from 453 patients with schizophrenia spectrum disorders, recruited from ten global sites, and clipped their speech recordings into 6,664 segments. Across three feature sets, acoustic-prosodic profile, pretrained multilingual embeddings, and their concatenation, we compared 16 algorithms to predict eight relapse-related PANSS items, including three positive (P1, P2, P3), three negative (N1, N4, N6), and two general (G5, G9) items, on speaker-disjoint splits (80% train, 10% test, and 10% validation). Performance was assessed by root-mean-squared-error (RMSE) at both segment and participant (median aggregation) levels. Best model per item underwent bias checks for age, sex, education, and symptom severity. Outcomes Best-performing models predicted symptoms with prediction errors of 1·5 PANSS points or lower: P1 1·494/1·527, P2 1·318/1·107, P3 1·407/1·542, N1 1·029/1·030, N4 1·452/1·430, N6 0·860/0·855, G5 0·850/0·882, G9 1·213/1·282 (segment/participant). Performance of the pretrained multilingual embeddings surpassed acoustic-prosodic features and their concatenation. Results were comparable in low-resource languages (e.g., Czech). We found no bias by age, sex, or education, aside from reduced N4 accuracy in males; but performance degraded with higher symptom severity. Interpretation Speech can support automatic assessment of schizophrenia symptoms using pretrained multilingual embeddings, even without the use of transcripts. Such models show promise as clinically meaningful, efficient, and low-burden tools for real-time monitoring of symptom trajectories. Funding EU Horizon research and innovation programme. Research in context Evidence before this study Automatic assessment of disease severity is a key issue in schizophrenia research, for which spontaneous speech offers a cost-effective, automatable solution. To evaluate existing evidence for speech-based symptom assessment, two reviewers (RHe, MK) searched PubMed, IEEE Xplore, arXiv, bioRxiv, and medRxiv for publications from inception to Aug 25, 2025, using the terms: (“symptom” OR “PANSS” OR “Positive and Negative Syndrome Scale”) AND (“psychosis” OR “schizophrenia”) AND (“language” OR “speech” OR “spontaneous speech”) AND (“prediction” OR “machine learning” OR “deep learning” OR “algorithm” OR “neural network” OR “AI” OR “artificial intelligence”). Fourteen studies on symptom-level modelling were identified. Ten studies dichotomized clinical scores (e.g., PANSS) into low vs high for classification: five used conventional ML (e.g., random forests) and five used neural networks, with F1 scores ranging from 0·60-0·85. The remaining four studies, and two of the ten studies as mentioned above, modelled raw scores directly as regression tasks. Two relied solely on conventional regressors and the rest used neural networks, with errors from 0·487 for single items (scale 1-7) to 8·04 for summed scores (scale 18-126). All studies used free speech for elicitation, except one study, which used a reading task. Three studies incorporated additional tasks, such as picture description and immediate recall. None were multilingual: nine were in English, three in Chinese, one in Swiss German, and one in Brazilian Portuguese. Features spanned a wide range, including acoustic-prosodic profiles, morpho-syntactic structure, semantic organization, pragmatics (including sentiments), and even visual features capturing movement during talking. Representations from pretrained language models were also widely employed. Sample sizes (counting patients with schizophrenia) were generally small: eleven studies enrolled