Enhanced recognition of dynamic indian sign language (ISL) hand signs using LSTM and mediapipe holistic frameworks

View at publisher Summarise with VIRAT
Authors: Divya Mohan, Chikku Elizabeth Shaji, Basil Philip, Mathew Steeve, Benoj Varghese Mathew
Cited by: 0
References: 7

Summary

No summary yet. VIRAT can read this paper and write a plain-language summary of its aim, methods, key findings and limitations.

Summarise with VIRAT Free to try with your Science Catalogs account.

Abstract

Sign language serves as an essential tool in enabling effective interaction for those who are deaf, possess hearing challenges, or lack verbal communication abilities, empowering them to actively participate in their surroundings. However, the shortage of widespread knowledge and learning of sign language among the general population often leads to social barriers, limiting employment and educational opportunities for those reliant on sign language. Addressing this challenge requires the development of efficient systems to facilitate communication between signers and non-signers, leveraging technological advancements such as machine learning. In this study, we propose our model MediaPipe-Optimised Long Short-Term Memory (MOPLSTM) model, a system utilizing LSTM (Long Short-Term Memory) and MediaPipe to address the classification of dynamic hand signs from the Indian Sign Language (ISL). LSTM networks excel in this domain because they possess the capability to understand the sequential patterns present in data over time, which makes them effective for interpreting the dynamic nature of sign language gestures. By leveraging Google’s MediaPipe Holistic framework, we captured and processed 15 distinct hand signs, achieving remarkable performance. This breakthrough showcases the potential of deep learning in bridging the communication gap between hearing and hard-of-hearing or deaf individuals.

Fields of Study & Subjects

Hand Gesture Recognition Systems Human Pose and Action Recognition Hearing Impairment and Communication

Publication details

Journal / source
AIP conference proceedings
Volume
3341
Pages
020028–020028
Publication date
1 January 2026
Language
EN