LMCM: Large Pre-trained Model Driven Multimodal Emotion Recognition via Cross-Modal Attention

With the continuous progress of deep learning technologies, emotion recognition has become an essential research topic in fields including HCI, intelligent assistants, and mental health applications. Compared to unimodal methods, emotion recognition across multiple modalities can offer a deeper insight into human affective states. However, existing methods still suffer from insufficient cross-modal feature alignment, limited generalization ability, and underutilization of large pre-trained models. To tackle these problems, this study proposes a new framework termed LMCM. For the visual modality, CLIP is employed as the feature extractor to leverage its strong image-text alignment capability; for the textual modality, the DeBERTa-V3-base model is adopted to obtain high-quality semantic representations. In the fusion stage, a residual cross-attention and a dual-branch parallel cross-attention mechanism are designed to maximize the use of complementary cues between visual and textual modalities. Experiments conducted on the IEMOCAP demonstrate that the method of this study surpasses earlier baselines, with higher Weighted Accuracy (WA) and Weighted F1-score (WF1). This study not only validates the potential of large-scale pre-trained models for multimodal emotion recognition but also provides a reproducible paradigm for future research on cross-modal feature fusion.