SST3D: spatially supervised sampling and temporally adaptive aggregation for 3D object measurement and detection

Abstract 3D object detection plays a vital role in autonomous driving and intelligent measurement systems. Transformer-based frameworks have recently gained prominence by predicting sampling points and directly extracting features from the image space. This design bypasses explicit depth estimation, bird’s eye view projection, and post-processing steps such as non-maximum suppression, thereby significantly improving inference speed. However, these methods often lag behind view transformation-based approaches in detection accuracy, particularly in the quantitative estimation of object size, orientation, and motion parameters. This paper attributes this performance gap to the difficulty of extracting reliable object features from the sampled locations. Since the sampling points are only indirectly supervised by detection losses, many of them often do not accurately correspond to actual object regions, resulting in the collection of background or irrelevant features. Furthermore, existing motion-based strategies for aligning sampling points across temporal frames often suffer from significant errors, which further compromise the quality of aggregated temporal features. To tackle these challenges, we introduce a novel 3D object detection and measurement framework that integrates two core components: point-driven sampling, which leverages supervision from point clouds to direct the sampling process toward object-relevant regions during training, thereby enhancing the fidelity of geometric measurements; and center-aligned temporal aggregation, which uses a learnable module to adaptively align and fuse temporal features, eliminating the need for explicit motion compensation while improving dynamic measurement accuracy. Experiments on the nuscenes benchmark further validate the effectiveness of our approach. Using ResNet-50, our method achieves 56.0 NDS, 45.9 mAP, and 38.4 ms latency, while the ResNet-101 variant reaches 59.5 NDS, 51.1 mAP, and 84.1 ms latency–achieving a favorable balance between accuracy and speed.