SignCraft: Transformative Learning-Based Sign Gesture Generation for Consumer Electronics

Indian Sign Language (ISL) is used by the deaf and the hard of hearing in India. But there is still a huge communication gap between ISL users and non-ISL users. To make learning and training more inclusive and accessible, this paper proposes SignCraft, an educational tool that provides hand gesture training by leveraging modern day consumer electronics. SignCraft proposes a hybrid architecture that combines GPT-3.5Enhancedlayer and a DFGANU–Netlayer. The GPT-3.5Enhancedlayer converts the real-time speech content from YouTube videos into sign gloss text. The DFGANU–Netlayer generates the corresponding gesture from the transcribed text. SignCraft integrates these models into an optimized on-device pipeline designed for deployment on consumer electronic platforms such as smartphones, tablets, and smart televisions, enabling low-latency, quantized inference and real-time execution on portable hardware that aligns with the design and accessibility priorities of modern consumer electronics. Experimental results show that the hybrid architecture of SignCraft is effective. The GPT-based transcription achieves an average Bilingual Evaluation Understudy (BLEU) score of 44.25. The trained DFGANU–Netmodel achieves an Inception Score (IS) of 19.59, Fréchet Inception Distance (FID) of 16.54, R-Precision of 80.62. The gestures that are generated are very realistic and look human-like.