Underwater sign language recognition using convolutional and transformer models: a comparative study in clear and murky conditions

Abstract Underwater communication is essential for divers, yet traditional methods such as hand gestures are often subject to misinterpretation, particularly under challenging visual conditions like murky water. In this study, we investigate the application of deep learning models for recognizing underwater sign language from static images, treated as an image classification task. We evaluate and compare two representative architectures: ResNet-18, a convolutional neural network, and a Vision Transformer, across two curated underwater datasets, representing clear and murky visual conditions. Experimental results show that both models achieve high accuracy in clear water, but the Vision Transformer exhibits significantly better generalization and robustness in murky water. Our findings show that it has potential for reliable underwater gesture recognition and human–robot interaction. To our knowledge, this work presents the first practical deep learning framework for underwater sign translation and discusses current limitations, including limited gloved-hand samples and challenges in accurate keypoint localization.