AI-BASED LIP READING RECOGNITION MODEL USING DEEP LEARNING

An AI-driven visual speech recognition system was created to precisely decipher spoken words from silent video by examining lip movements.The technology leverages deep learning architectures, combining Conformer and Transformer layers with ResNet-based visual feature extraction, to capture intricate spatiotemporal patterns necessary for lip reading.To overcome the limitations of manually labelled datasets, the system incorporates large-scale unlabeled video datasets like AVSpeech and VoxCeleb2, where automatic transcriptions are produced utilizing sophisticated ASR models like Whisper, Wav2Vec2.0,HuBERT, and Conformer-Transducer, in order to get beyond the limits of manually annotated datasets.This scalable automatic labeling pipeline improves model robustness across a variety of speakers and contexts and greatly increases training data.The system performs well even with noisy labels, achieving improved word error rates on benchmark datasets such as GRID and LombardGrid.Using cutting-edge deep learning techniques, the solution provides an effective, data-driven, and highly generalizable method of lip reading for media applications, accessibility, silent communication, and surveillance.