A Radar and Camera Cross-Modal Fusion Attention Method for Foreign Object Debris Detection

Foreign Object Debris (FOD) refers to a type of foreign substance on an airport runway that may damage aircraft, such as metal parts, golf balls, caps, fuel tank covers, pipes, etc. The existing detection systems apply millimeter wave (MMW) radar and camera to detect FOD, which work independently and cannot effectively combine the advantages of the two sensors. In this work, a novel Fusion Edge Light Instrument is developed for FOD detection, the instrument adopts a special radar frequency band (92-94GHz) in the design, which is particularly convenient for deployment in the airport. A network (FFOD-Net) is proposed to integrate the information from the instrument. In order to enhance the performance of FOD small object detection, a feature extraction module is built, which applies multiple receptive fields and channel attention to process MMW radar and camera data in parallel. Afterward, a cross-modal attention with dynamic enhancement of features and multiscale detection module is applied to integrate information from the two sensors effectively. Moreover, the FOD dataset is collected using airport runway fusion edge light instrument, and experimental verification is performed on this dataset. Based on experiments, the proposed method achieves the best mean Average Precision (mAP@0.5) of 80.55%, which indicates that the method effectively integrates information from fusion edge light and exhibits superior detection performance compared to existing fusion approaches.