Dynamic Unified Alignment with Low-Text Dependency for Multimodal Sentiment Analysis

Multimodal sentiment analysis has become a research hotspot in the field of human-computer interaction, aiming to fuse speech, vision, and text for user emotion recognition. However, existing approaches commonly rely on textual modality as one of the dominant information sources, leading to poor robustness and insufficient generalization ability in scenarios where text is missing or of low quality. To address this issue, we propose the dynamic unified alignment for low-text dependency (DUAL) model, which prioritizes audio and visual modalities while leveraging the textual modality as a means of semantic refinement, thereby effectively mitigating the over-reliance on text. In this model, we construct a unified emotional feature space and perform modality alignment for audio and visual inputs via nonlinear mapping. Furthermore, we introduce the sentiment differential module (SDM) and the visual differential module (VDM), which extract emotional change trends from the audio and visual modalities by calculating feature differences across consecutive audio segments and video frames. Additionally, we design a cross-modal sentiment residual (SR) mechanism that maps one modality into the feature space of another and computes their residuals to reinforce complementary information between modalities. During the decoding phase, textual features are introduced with delay and utilized via a cross-attention mechanism for semantic refinement, enhancing the model's ability to capture semantic details while reducing sensitivity to textual quality. Moreover, we construct a joint optimization objective composed of classification loss, static alignment loss, and dynamic alignment loss, which not only maintains high recognition accuracy but also improves cross-modal expression consistency. Experimental results on several public datasets demonstrate that the DUAL model maintains high sentiment recognition accuracy while exhibiting greater robustness and stability in scenarios with missing or noisy text.