AI for Inclusion: Sign Language Recognition to Support Hajj Pilgrims

Hajj pilgrimage is not only performed by normal people, but can also be performed by people with disabilities, such as deaf people; therefore, this study proposes a deep learning approach for sign language recognition, specifically Indonesian Sign Language (BISINDO), using two leading object detection models, YOLOv8 and YOLOv11, for inclusive applications for disabled hajj pilgrims. The dataset used in this study consists of images representing hand gestures in BISINDO, which is designed to recognise nonverbal communication from individuals with disabilities who have hearing and speech impairments. The YOLOv8 and YOLOv11 models are trained to detect and classify hand gestures used by hajj pilgrims, thus enabling better interaction during Hajj. The results show that YOLOv8 achieves higher detection accuracy than YOLOv11, resulting in a significant increase in the mean average precision (mAP) across training periods. In the 60th period, YOLOv8 achieved an mAP of 0.963, while YOLOv11 recorded an mAP of 0.960. This study demonstrates that the application of YOLO-based object detection technology in the context of BISINDO iconic language recognition can enhance accessibility and communication for disabled pilgrims, thereby improving their spiritual experience. This approach offers an inclusive solution that can be integrated with support systems in the Holy Land.