Efficient Deep Q Learning Models Featuring Adaptive Attention and Emotionally Augmented Fusion for Multimodal Sentiment Analysis

Abstract— The interplay of the features of text, audio, and video makes multimodal sentiment analysis very critical in psychological assessments. Traditional methods have failed to prioritize relevant features well in order to capture the rich emotional dynamics that are central to most psychological insights. Most limitations associated with existing methods include not having a proper modality-specific focus on relevance and poor robustness to noisy inputs along with the lack of interpretability sets. In this direction, we introduce an integrated framework with three novel deep Q-learning models that could enhance multimodal sentiment analysis. These are Adaptive Attention-Gated Deep Q-Learning (AADQL), Emotionally Augmented Reinforcement Fusion Network (EARFN), and Hierarchical TaskAware Deep Q- Learning (HTADQL). AADQL incorporates the adaptive attention gating mechanism in order to dynamically weigh the relevance of features across the different modalities, and we found it to result in improved accuracy of 4-6% and reducing computational overhead by 15-20%. EARFN employs psychological emotion models to combine emotion- specific representations within a reinforcement learning paradigm, and it produces high robustness with 25-30% noise tolerance and superior prediction accuracy of 88- 90%. HTADQL makes use of a hierarchical architecture, which solves the subtasks of valence and arousal before combining them for sentiment classification, thus showing better interpretability and error rate reduction by 30-40%. These methodologies hold some significant implications: they improve the accuracy up to 10% against baseline models; exhibit robustness in noisy environments and deliver increased interpretability about the psychological sentiment dynamics. The work that is reported here brings in a remarkable advancement on the deep Q-learning front for multimodal sentiment analysis and would be even more apt to create room for even more efficacious psychological assessment tools.