Effective selective residual fusion method for occlusion-aware object detection in traffic investigations

Abstract With advances in measurement instruments, extracting object information from traffic camera videos constitutes a crucial component of traffic investigations. However, numerous object features in such videos are often obscured by instance overlap and/or rain interference (RI), which substantially degrades detection accuracy. To address this issue, we propose an object detection method termed selective residual fusion, designed to enhance the detection of occluded traffic objects under overlapping and/or rainy conditions. In this method, the selective discriminative analysis module with joint spatial-frequency domain is presented to improve video quality adaptively. The deep residual network is adopted as the backbone network to extract features. The improved feature fusion structure is presented to enhance the detection of occluded traffic objects. The density estimation structure is presented to analyze the overlapping region, and can automatically adjust thresholds in complex and changeable traffic scenes. Different comparative experiments are conducted, and the effectiveness of our presented method is verified. When detecting occluded traffic objects, our presented method can achieve 93.3% accuracy for detecting fully-visible objects, 91.7% accuracy for detecting partly-occluded objects, and 88.9% accuracy for detecting difficult-to-see objects on visual object classes (VOC) data; 96.6% accuracy for detecting fully-visible objects, 94.6% accuracy for detecting partly-occluded objects, and 92.3% accuracy for detecting difficult-to-see objects on Karlsruhe institute of technology and Toyota technological institute (KITTI) data; 80.4% accuracy for detecting fully-visible objects, 66.3% accuracy for detecting partly-occluded objects, and 63.8% accuracy for detecting difficult-to-see objects on Berkeley deep drive 100k (BDD100K) data; 92.3% accuracy for detecting fully-visible objects, 90.3% accuracy for detecting partly-occluded objects, and 87.0% accuracy for detecting difficult-to-see objects on RI version 1 (RIv1) data; 90.3% accuracy for detecting fully-visible objects, 85.2% accuracy for detecting partly-occluded objects, and 84.7% accuracy for detecting difficult-to-see objects on RI version 2 (RIv2) data. And our presented method can achieve an efficiency of 62.1 frames s −1 . The results show that our presented method has excellent detection performance, especially for occluded traffic objects caused by instance overlap and/or RI. Hence, our presented method has applicability for traffic investigations under overlap and/or rain-induced occlusions.