Sign Lingo: Live Sign Language Interpretation using AI

Sign language serves as the primary mode of communication for deaf and mute individuals, enabling the expression of needs and emotions. However, the widespread lack of sign language knowledge among hearing individuals creates significant communication barriers in both social and professional environments. Speech impairments further restrict oral and auditory interactions, highlighting the urgent need for effective assistive technologies. This research proposes an advanced speech- and text-to-Indian Sign Language (ISL) conversion system utilizing 3D modelling techniques to generate realistic sign gestures. A lightweight Convolutional Neural Network (CNN) was developed for sign recognition, achieving a high accuracy of 95.6%. Additionally, the system employs 3D-modeled videos to visually render ISL gestures, reaching a gesture rendering accuracy of 99.2%. The system demonstrates wide range of applications in many fields including daily communication, business meetings, classroom education, and speech-based interactions. The paper ends with recommendations for future work, recommending algorithm optimization, the expansion and diversification of datasets, and a user-centric approach to design to further improve usability and drive broader adoption of such assistive communication technologies.