Silent video to text: CNN-based deep learning model for lip movement recognition and transcription

Lip reading makes it possible to understand spoken language by visually recognizing lip movements, which is important for use in those who are deaf or in noisy environments. The study introduces a machine learning-based algorithm for interpreting lip movements into text using convolutional neural networks (CNNs). The tool is constructed as an online Django web application used to convert speech to text from video sources within assistive communication. The application will extract the video frames and preprocess each frame to enhance the model&s;s learning and training and will split these frames in a test and train data set. The convolutional neural networks will learn the lip movement patterns, which will allow the model to recognize spoken word. The model will be assessed with precision, recall, F1 score, and confusion matrix to evaluate the models classifications and misclassifications.