Multilingual video dubbing system
Summary
No summary yet. VIRAT can read this paper and write a plain-language summary of its aim, methods, key findings and limitations.
Summarise with VIRAT Free to try with your Science Catalogs account.Abstract
The exponential expansion of digital multimedia across international platforms necessitates efficient multilingual dubbing solutions. Conventional dubbing methodologies prove resource-intensive and economically prohibitive for widespread content localization. This research introduces an intelligent automated dubbing framework integrating advanced neural architectures for speech processing, translation, and synthesis. The system employs Whisper for acoustic modeling, NLLB-200 for cross-lingual translation, XTTS v2 for voice cloning, and Wav2Lip GAN for visual synchronization. A novel segment-based processing approach ensures temporal precision between synthesized audio and source video. Experimental validation demonstrates superior naturalness and synchronization accuracy compared to existing methodologies. The framework addresses critical applications in educational technology, digital entertainment, corporate communication, and accessibility enhancement.
Fields of Study & Subjects
Speech Recognition and Synthesis
Speech and Audio Processing
Generative Adversarial Networks and Image Synthesis
Publication details
- Journal / source
- Zenodo (CERN European Organization for Nuclear Research)
- Publication date
- 27 March 2026
- Language
- EN