Transfer Learning-Based Recognition of Bhutanese Sign Language Digits Using Deep CNNs

The hearing and speech-impaired community uses hand gesture-based communication media to communicate with general public. However, the general public finds it difficult to communicate with them due to their difficulties in understanding sign digits, thereby creating a communication gap between the general public and the hearing-impaired community. Therefore, this paper proposes three pre-trained (VGG16, ResNet50, MobileNet) models to train models on the Bhutanese sign digit dataset. In this study, two different datasets (Bhutanese sign digit and Turkish sign digit) were merged and used to train the models. The rationale for merging the datasets is that both datasets use the same representation of sign gestures and have different variations of images. The dataset was split into train and test sets with a ratio of 80:20. The VGG16 network architecture outperformed the other two models with the training and testing accuracy of 96.72% and 95.85%. The trained model was integrated with the Django framework to create a web application for digit recognition.