Assessing the efficacy of deep learning approaches for facial expression recognition in individuals with intellectual disabilities

Abstract Purpose Facial expression recognition has gained significance as a means of providing artificial agents with the capacity to discern the emotional states of users. Notable performance has been achieved by deep learning approaches; however, its direct use for recognizing facial expressions in individuals with intellectual disabilities has not yet been studied in the literature, to the best of our knowledge. The aim of this work is to thoroughly explore the applicability of these systems to people with intellectual disabilities. Methods To address this objective, we work with a set of 12 convolutional neural networks (CNNs) in different approaches, including an ensemble of datasets without individuals with intellectual disabilities and a dataset featuring such individuals. Further, we apply the explainable artificial intelligence technique LIME that allows understanding the important regions considered by the CNN. Results Our examination of the outcomes, both the performance and the important image regions for the models, reveals significant distinctions in facial expressions between individuals with and without intellectual disabilities, as well as among individuals with intellectual disabilities. Conclusions Our findings show the need of facial expression recognition within this population through tailored user-specific training methodologies, which enable the models to effectively address the unique expressions of each user.