Multi-element animation generation via memory-augmented self-supervised learning and mixture density networks

To address the challenges of motion uncertainty, difficulty in modelling temporal dependencies, and poor scene consistency in multi-element animation generation, this paper proposes a multi-element animation generation model based on memory-enhanced self-supervised learning and a mixture of density networks.The model first employs a dual-channel convolutional neural network to extract features from animation video frames and linguistic signal data, respectively, and fuses these representations through self-supervised learning tasks.It then utilises a memory matrix to store and recall long-term contextual information, yielding enhanced features.These enhanced features are then fed into a MDN to obtain the probability distribution of the data.Finally, upsampling reconstructs the animation video frames.Comparisons with video diffusion models, StyleInV, AnimateDiff, and DaGAN demonstrate improvements of 6.6, 3.8, 13.1, and 2.3 in FID score, respectively, indicating MEM-MDN's superior capability in generating multi-element animations.