UniSparseBEV: A Multi-Task Learning Framework With Unified Sparse Query for Autonomous Driving

Recent advancements in vision-centric multi-task learning have greatly impacted autonomous driving, with a focus on constructing efficient and rich Bird’s Eye View (BEV) representations. While these methods achieve impressive performance, they often suffer from structural complexity and high computational costs due to the need for dense BEV representations. To address these challenges, we propose UniSparseBEV, a simple and efficient vision-based multi-task learning framework based on sparse queries. We introduce a set of learnable shared queries to facilitate information exchange across tasks. Additionally, we propose theZ-axis Deformable Cross-Attention (Z-DCA) module, which enables BEV segmentation task queries to directly extract information from image features without requiring dense BEV representations. To further enhance training efficiency, we incorporate 2D supervision into the network. Extensive experiments on the NuScenes dataset demonstrate that UniSparseBEV outperforms existing single-task methods in 3D object detection and BEV segmentation. A detailed robustness analysis is also conducted on the UniSparseBEV framework. We hope UniSparseBEV can serve as a strong baseline for multi-tasking in autonomous driving.