MFF-PEA: an automatic assessment model for professional spoken English via multimodal feature fusion

Accurate assessment of professional spoken English necessitates capturing nuanced linguistic accuracy and non-verbal paralinguistic cues in cross-cultural communication settings.To address limitations of unimodal approaches and static fusion methods, we propose Multimodal Feature Fusion-based Professional English Assessment (MFF-PEA), an adaptive framework integrating speech, facial expressions, and gestural dynamics.The core innovation lies in a cross-modal dynamic fusion (CMDF) mechanism that employs learnable attention gates to weight modalities based on contextual relevance.For joint optimisation, a hybrid loss function combines regression loss for absolute scoring and pairwise ranking loss for proficiency discrimination.Rigorous evaluations on multi-domain professional datasets confirm MFF-PEA's significant superiority over state-of-the-art baselines, exhibiting stronger predictive consistency and lower assessment errors.Comprehensive ablation studies validate each architectural component's necessity, while cross-domain tests in business, medical, and legal scenarios demonstrate transferable robustness.This work establishes a context-sensitive paradigm for automated multimodal language assessment.