Video-Based Sign Language Recognition Using CNN for Frame Feature Extraction and a Hybrid Temporal Model with Self-Attention and LSTM

View at publisher Summarise with VIRAT
Authors: Sakshi Kharat, Bhakti Palkar, Mansi Kambli
Cited by: 0

Summary

No summary yet. VIRAT can read this paper and write a plain-language summary of its aim, methods, key findings and limitations.

Summarise with VIRAT Free to try with your Science Catalogs account.

Abstract

The main form of communication for hearing-impaired individuals is sign language, although non-signers face challenges in understanding it. To bridge this gap, we propose a hybrid model designed to recognize sign gestures using the LSA64 dataset. Our approach employs a custom 3D-CNN to extract spatiotemporal features, integrated with a hybrid sequence model to capture temporal dynamics. This sequence model combines Temporal Convolutional Layer, Self-Attention Mechanism for frame-to-frame relationships, and a LSTM network for modeling long-term dependencies. We compared our model against six-pretrained feature extractors, all using the same sequence modeling architecture. The experimental results indicate that our proposed model surpasses all the baselines with an accuracy of 90.31% thereby demonstrating its effectiveness in recognizing sign gestures.

Fields of Study & Subjects

Hand Gesture Recognition Systems Human Pose and Action Recognition Hearing Impairment and Communication

Publication details

Journal / source
CRC Press eBooks
Pages
6–12
Publication date
25 March 2026
Language
EN
Document type
Book chapter