Real-Time Bidirectional ASL Alphabet Recognition Using Hand Landmarks and a BiLSTM-Attention Network with Confidence-Gated Decoding

We present a real-time American Sign Language (ASL) fingerspelling system for recognizing the static alphabet (A-Z) from webcam video and rendering typed text back into sign exemplars. For sign-to-text, we extract 21 hand landmarks per frame using MediaPipe Hands (63-D (x, y, z) features) and classify 30-frame landmark sequences using a stacked bidirectional LSTM (BiLSTM) with a temporal attention mechanism. To stabilize real-time decoding, predictions are committed only when the top-class probability exceeds 0.9 and the same class persists for 12 frames. To prevent temporal overlap leakage, recordings are split in a session-disjoint manner before sliding window sequence generation. All results in this paper are based on a single signer and therefore reflect generalization to unseen recording sessions of the same signer (not cross-signer generalization). On a 26-class dataset of 2520 sequences (train/val: 1977/543), the model achieves 90.42% validation accuracy with Top-3 accuracy 97.42%. Runtime performance is benchmarked on Apple M3 (Metal), 16 GB RAM at 17.3 FPS for the full pipeline with mean end-to-end latency 57.75 ms. Errors are concentrated in visually similar handshapes (e.g., S→E, T→N). We discuss limitations (dynamic motion for J/Z, single-hand assumption, and single-signer data) and directions for robust, mobile, and sentence-level sign understanding.