Text-to-video generation with multi-loss sequential generative adversarial network

Read full text Summarise with VIRAT
Authors: Anwar Ullah, Zhang Xing, Md All Moon Tasir, Mohd Nor Akmal Khalid, Abdul Majid, Naresh Kumar, Herbert Mukalazi
Cited by: 0
References: 46

Summary

No summary yet. VIRAT can read this paper and write a plain-language summary of its aim, methods, key findings and limitations.

Summarise with VIRAT Free to try with your Science Catalogs account.

Abstract

Generating videos from text remains challenging due to issues like structural complexity, semantic alignment, temporal coherence, and frame discontinuity. It is still highly difficult to generate realistic videos where the frames must comply to both spatial and temporal coherence, despite the fact that Generative Adversarial Networks (GANs) have been successfully deployed to generate videos conditioned on a text description. This study proposes a Sequential Generative Adversarial Network (Seq-GAN) that employs multiple generators and discriminators to generate realistic and high-quality videos using written descriptions, while maintaining semantic alignment and relevance between frames. The approach incorporates a modified Variational Autoencoder (VAE) with reduced and adjusted 2D layers to extract static “gist” features for improved performance. For dynamic frame generation, the model uses three generators ([Formula: see text], [Formula: see text], [Formula: see text]) and three discriminators ([Formula: see text], [Formula: see text], [Formula: see text]) trained with different optimizers: RMSprop, Adam, and SGD. [Formula: see text] and [Formula: see text] generate basic frame structures, while [Formula: see text], [Formula: see text], [Formula: see text], and [Formula: see text] refine video quality and ensure relevance to the text. Experimental results on SBMG, KTH, and UCF-101 datasets show that Seq-GAN achieves notable improvements in Inception Score (IS), Fréchet Video Distance (FVD), and CLIP-similarity confirming the method’s effectiveness in generating realistic, semantically accurate video sequences.

Fields of Study & Subjects

Generative Adversarial Networks and Image Synthesis Multimodal Machine Learning Applications Digital Media Forensic Detection

Publication details

Journal / source
Scientific Reports
Volume
16
Issue
1
Publication date
25 March 2026
Language
EN