A Real-time Framework for American Sign Language Recognition Using Hand Landmarks and Machine Learning

Communication is the act of sharing information, ideas, or feelings. For effective two-way communication, messages must be conveyed and interpreted with a common language, typically through verbal or non-verbal means such as speech, gestures, or body language. When this shared understanding is absent, communication can break down. For deaf or hard-of-hearing (DHH) individuals, communication barriers can lead to social isolation and limited opportunities. This research aims to overcome these barriers by developing an American Sign Language (ASL) detection tool using machine learning to recognize and interpret ASL gestures, with potential to enhance education, employment, and everyday communication for people with hearing impairments. The system is built around Python and leverages several frameworks and libraries, including TensorFlow, OpenCV, scikit-learn, and MediaPipe. MediaPipe provides real-time hand tracking, OpenCV handles video processing, and a scikit-learn-based classifier (Random Forest) maps hand landmarks to ASL gestures, resulting in robust performance with real-time capability. The approach emphasizes effective preprocessing and feature extraction from MediaPipe hand landmarks to achieve high accuracy (96.5%) while maintaining low latency. Despite challenges such as gesture misclassifications, lighting variations, and occlusions, the system demonstrates resilience in detecting hand signs. Future enhancements will explore deep-learning architectures, adaptive preprocessing to handle variable illumination, and larger, more diverse datasets to improve accuracy and generalization. Overall, this work contributes to real-time sign-language recognition and broadens access for the DHH community.