An Explainable Multi-Model Framework for Detecting Self-Stimulatory Behaviors in Real-World Autism Videos

Identification of Autism Spectrum Disorder (ASD) Autism Spectrum Disorder Early intervention for Autism Spectrum Disorders (ASD) needs early diagnosis; the problem is, though, that the process of early diagnosis remains challenging and is based on subjective judgment. Autistic Stereotypies consist of actions such as hand flapping and spinning. These actions are typical of yet not extensively researched as markers of ASD. These actions occur in daily situations and can be recorded via a smartphone. This manuscript presents an automatic, objective, and interpretation-ready approach for the detection and classification of such patterns in unrestrained videos. By resorting to the only publicly available benchmark dedicated to full-body repetitive patterns in unconstrained environments—the Self-Stimulatory Behaviour Dataset (SSBD)—we assess the performance of three different techniques: the Extended Bag-of-Visual-Words method employing Space-Time Interest Points, the hybrid CNN-LSTM network architecture, and the fine-tuned Inflated 3D Convolutional Network (I3D). The proposed models perform greatly on the benchmarked task and deliver superior results to all former baselines. In addition to that, SHAP and LIME methods of Explainable AI assist in giving spatiotemporal explanations specific to behaviors. The proposed system offers a non-invasive scalable solution to preliminary screening of ASD on a smartphone