Silent2Sound: Visual-to-Audio Speech Reconstruction Using Lip-Reading AI

Lip reading, or visual speech recognition, identifies spoken words by analyzing only the movements of the lips and surrounding facial region, making it valuable when audio is noisy or unavailable.This project implements a deep learning-based system that recognizes speech from video using two components.The first is an end-toend visual speech recognizer trained on the GRID corpus, where mouth regions are extracted using dlib landmarks and processed through a 3D CNN and Bi-GRUs, with CTC loss enabling character-level transcription from variable-length frame sequences.The second component enables real time phrase-level recognition by using only normalized mouth landmark coordinates, which are classified by a lightweight LSTM model to detect predefined phrases.A Flask-based web interface connects both components, allowing users to upload videos or use their webcam, with the backend handling preprocessing, model inference, and real-time display of the recognized text.