Multiview BEV Fusion From Vehicle-on-Board and Roadside Cameras for 3-D Object Detection

3D object detection, a sought-after feature for connected and autonomous vehicles (CAVs), offers precise localization and classification of obstacles by estimating their spatial attributes from 2D images. This approach addresses the limitation of traditional 2D detection, which lacks spatial information. However, the task becomes challenging in complex environments with occlusions and restricted line-of-sight views, such as intersections and winding roads. Fortunately, roadside infrastructure equipped with traffic cameras, commonly deployed at these critical locations, offers a wider field of view. It can therefore provide valuable support to CAVs and improve object detection performance. Vehicle-to-infrastructure (V2I) cooperative 3D detection frameworks leverage vehicle-to-everything (V2X) communication to transmit roadside data to CAVs. By integrating multi-view observations from both vehicle-mounted and roadside cameras, these frameworks enable a more accurate and semantically rich understanding of the driving environment. In this paper, we propose V2IFormer, a multi-view feature fusion framework based on BEV representations, designed to enhance 3D object detection. Specifically, V2IFormer extracts semantic features from 2D images captured by both vehicle-mounted and roadside cameras and transforms them into BEV features. At the CAV side, a deformable mutual-attention algorithm is employed to dynamically select spatial features and fuse the most informative BEV representations from both sources. This strategy reduces interference from irrelevant or corrupted features and improves fusion efficiency. Finally, we validate the effectiveness of V2IFormer on the DAIR-V2X benchmark. Our framework achieves state-of-the-art 3D object detection performance, particularly in scenarios with occlusion and limited line-of-sight.