Multilingual video dubbing system

Read full text Summarise with VIRAT
Authors: Vijaya Lakshmi M, Mounika L, Bala Sreeman Sai G, Shanmukha Veera Brahmam J
Cited by: 0

Summary

No summary yet. VIRAT can read this paper and write a plain-language summary of its aim, methods, key findings and limitations.

Summarise with VIRAT Free to try with your Science Catalogs account.

Abstract

The exponential expansion of digital multimedia across international platforms necessitates efficient multilingual dubbing solutions. Conventional dubbing methodologies prove resource-intensive and economically prohibitive for widespread content localization. This research introduces an intelligent automated dubbing framework integrating advanced neural architectures for speech processing, translation, and synthesis. The system employs Whisper for acoustic modeling, NLLB-200 for cross-lingual translation, XTTS v2 for voice cloning, and Wav2Lip GAN for visual synchronization. A novel segment-based processing approach ensures temporal precision between synthesized audio and source video. Experimental validation demonstrates superior naturalness and synchronization accuracy compared to existing methodologies. The framework addresses critical applications in educational technology, digital entertainment, corporate communication, and accessibility enhancement.

Fields of Study & Subjects

Speech Recognition and Synthesis Speech and Audio Processing Generative Adversarial Networks and Image Synthesis

Publication details

Journal / source
Zenodo (CERN European Organization for Nuclear Research)
Publication date
27 March 2026
Language
EN