BioFusionNet: Cross-modal EMG–speech Representation Learning Toward Motor–speech Severity Assessment in Cerebral Palsy

Cerebral palsy (CP) is a neurodevelopmental condition in which early brain injury can affect both motor coordination and speech production, making objective severity assessment important for rehabilitation planning.Clinical grading is often observation driven, which can introduce variability across sessions and raters.Prior computational studies using a single modality, such as electromyography (EMG) or speech, can capture only part of this impairment profile and may not transfer well across subjects and recording conditions.This paper presents BioFusionNet, a multimodal deep learning framework that integrates EMG and speech representations to study severity-related patterns through cross-modal fusion.The model combines an InceptionTime encoder for EMG feature learning, a Conformer encoder for speech analysis, and a Cross-Modal Transformer (CMT) that aligns temporal dependencies across modalities.Experiments use NinaPro DB1 for structured EMG activity signals and TORGO for dysarthric speech with corpus-defined severity labels, and the two modalities are treated as unpaired public sources.Since these datasets do not provide CP-specific clinical ratings such as GMFCS, the results are reported as an exploratory multimodal severity proxy study rather than direct clinical CP severity prediction.Under the defined evaluation protocol, BioFusionNet achieves an accuracy of 0.993, a macro F1-score of 0.982, and a Cohen's kappa of 0.968, exceeding unimodal and early-fusion baselines.The findings suggest that adaptive cross-modal alignment can capture complementary neuromuscular and articulatory cues, motivating future validation on clinically rated CP cohorts with documented label provenance and rater agreement.