Multimodal Feature Interactive Network for Machinery Fault Diagnosis Under Small Samples

Vibration sensing and infrared thermal image technology have been widely used in the health monitoring of machines. Multimodal fault diagnosis combining vibration and infrared thermal data has shown considerable potential. However, due to the heterogeneity of multimodal data, the performance of some models will significantly decrease under small samples. It is challenging to implement fault diagnosis based on multimodal data under small samples. Thus, a new DNN, i.e., adaptive multimodal feature interactive network (ADMMFI), is proposed in this study, where a multimodal feature interactive (MMFI) module is proposed to perform adaptive feature fusion under small samples. Firstly, a dimension reshaping module is proposed in ADMMFI to preserve discriminative thermal features during modality transformation. Secondly, a multimodal feature interaction module (MMFI) dynamically separates features into private and shared components, enabling adaptive fusion across modalities. Finally, the experimental results on a rolling bearing test bench and a rotor system test bench show that ADMMFI has a good performance on multimodal data fusion and feature extraction for machinery fault diagnosis under small samples. The fault recognition accuracy of ADMMFI on the two multimodal datasets was 79.17% and 95.15%, respectively. It demonstrates the effectiveness of ADMMFI compared with other DNNs.