HP-C4D: A Fast Camera and 4D Radar Fusion Framework With Height Prediction for 3D Object Detection

4D millimeter-wave radar, as a fundamental sensor for 3D object detection, has gained increasing attention in autonomous driving due to its robustness and additional elevation information. However, the sparsity and noisiness of 4D radar point clouds hinder its broader application. Fusing camera with 4D millimeter-wave radar provides an affordable and robust solution. In this paper, a fast and effective framework HP-C4D for camera and 4D radar fusion is proposed. Three key modules are proposed in HP-C4D. Firstly, we innovatively propose the Height BEVPool to predict the height of each BEV grid with negligible delay increase during the height compression. The predicted height information is incorporated into the final object height prediction. Secondly, Perceptive View Heatmap Guidance (PVHG) is proposed to suppress background noise's generation in image BEV and assist the Height BEVPool for better height prediction. Thirdly, Interactive Guidance Fusion (IGF) is proposed to efficiently fuse image BEV and 4D radar BEV. Extensive experiments on the View-of-Delft (VoD) and TJ4DRadset datasets demonstrate the effectiveness and advance of our proposed method. It is worth highlighting that the 3D mean average precision of our proposed method is 0.71% and 3.85% higher than the latest baseline method LXL (LiDAR-Excluded-Lean) on the VoD and TJ4DRadset datasets, respectively. To the best of our knowledge, HP-C4D is the fastest method for camera and 4D radar fusion in 3D object detection on the VoD dataset, achieving 21.4 FPS on single NVIDIA RTX 3090 GPU. Code will be released at https://github.com/c-yyyy/HP-C4D.