Graph-Based Representation Learning With Beta Uncertainty for Enhanced Multimodal Emotion Recognition

The performance of multimodal emotion recognition (MER) systems critically depends on their ability to effectively integrate information from heterogeneous sources. However, existing fusion methods including attention-based mechanisms, auxiliary regression, and sampling-based approaches often lack efficient strategies for managing uncertainty in the presence of noisy or conflicting inputs across modalities. To address this limitation, we propose a novel framework grounded in an Uncertainty-Aware Fusion Mechanism. Our system employs a Mixture-of-Experts (MoE) architecture with specialized Transformer encoders to learn modality-specific representations. Within the fusion module, each modality expert predicts the parameters of a Beta distribution, yielding a bounded reliability score within the [0, 1] interval. In contrast to computationally intensive sampling techniques or generic Gaussian-based models, our method enables efficient, single-pass confidence estimation. These Beta-derived reliability scores dynamically modulate the contribution of each modality, attenuating the influence of less reliable sources. The resulting uncertainty-weighted representation is subsequently refined by a Dynamic Graph Convolutional Neural Network (DGCNN), which captures high-order, sample-specific relationships among modalities. We evaluate our approach on three diverse benchmarks: the DEAP dataset for physiological, the MELD and IEMOCAP datasets for conversational emotion recognition. Experimental results demonstrate significant improvements over state-of-the-art methods, highlighting the effectiveness of the Beta-distribution-based fusion strategy for robust emotion recognition.