Arayvideo: text-to-video diffusion models with a transformer architecture

Text-to-video conversion technology is developing rapidly. Improving the quality and efficiency of video generation has become a key research topic. This has significant implications for promoting multimodal learning and automating video content creation. Current technology has deficiencies in richness of video frame details and video timing consistency, resulting in a lack of realism and smoothness in generated videos. To address these challenges, we propose an ArayVideo model. This model combines 3D Pyramid Aggregated Variational Autoencoder (3D-PAVAE) and 3D ArayVideo Transformer Architecture. The combination of 3D-PAVAE's multi-scale feature processing and 3D ArayVideo transformer's spatiotemporal dynamic capture significantly improves the quality of video content and inter-frame coherence. Experimental results show that compared with existing methods, ArayVideo improves VBench video quality evaluation index by 8.5%. Our method has significantly improved the detail richness and temporal coherence of videos, demonstrating its application potential in the field of video generation.