A Systematic Review of Deep Learning Approaches in Multimodal Emotion Recognition

With the rapid development of artificial intelligence, multimodal emotion recognition (MER) has become an important research area. Its main goal is to enable machines to understand and interpret human emotional states by integrating audio, text, and visual signals. This systematic review follows PRISMA guidelines and analyzes 62 high-quality studies published between 2020 and 2025, providing a comprehensive overview of deep learning-based MER methods. The results indicate that the audio modality is the most frequently utilized, appearing in all reviewed studies, whereas text and visual features are commonly employed alongside it to provide semantic and nonverbal information that together form the foundation for multimodal emotion interaction. Methodologically, the field is moving toward an end-to-end deep learning paradigm. Transformer models are often used to extract text features, whereas CNNs remain the primary choice for visual features. When combining different modalities, intermediate fusion methods with attention mechanisms are commonly employed. These methods help capture interactions between modalities more effectively. The IEMOCAP, CMU-MOSEI, and MELD datasets are widely used as standard benchmarks in MER studies. Even with these developments, MER research faces several challenges. Models can be computationally demanding, aligning different types of data remains difficult, and most systems do not generalize well across languages. This review summarizes the main strengths and weaknesses of current methods. It also provides guidance for researchers to address key technical issues, improve model designs, and make MER systems more practical and adaptable to real-world scenarios.