Automatic and explainable assessment for Parkinson’s disease by video-based human motion understanding

BACKGROUND: The assessment of Parkinson’s disease depends heavily on neurologist experience and involves significant clinical workload, where standard motor examinations require substantial time and face-to-face evaluation, limiting accessibility for patients with mobility constraints. Furthermore, current rating scales contain subjective definitions that lead to rating inconsistencies among clinicians. Existing automated methods exhibit severe problems, including being applicable to only single symptoms, lacking clinical interpretability, and insufficient accuracy. METHODS: We proposed an AI-based, fully automatic, and explainable PD assessment technique using videos. The system detects keypoints on face, body, hands, and feet, then extracts motion features including amplitude, frequency, velocity, and acceleration that directly correspond to MDS-UPDRS rating criteria, enabling explainable assessment. We evaluate all 16 vision-based items in the MDS-UPDRS motor examination, covering symptom categories such as masked face, bradykinesia, postural instability, and tremor symptoms. RESULTS: Our automated system achieved accuracies of 97.2%, 90.1%, 96.6%, and 96.2% for the four symptom categories respectively in our clinical experiments. When used as clinical support in the real clinical assessment process, the system improved clinician accuracy from 78.7 to 85.3% overall, with correction rates of 15.1%, 8.0%, 36.9%, and 80.1% for each symptom. CONCLUSIONS: This AI-based video assessment provides accurate, automatic, objective, and interpretable Parkinson’s disease evaluation while supporting clinical decision-making. The system enables remote monitoring and reduces healthcare resource strain, particularly benefiting regions with limited neurological expertise.