Toward standardized testing of automotive LiDAR: year 3 methods and results

This research collected data in a field test relevant to benchmarking automotive-grade LiDARs, with the goal of providing data toward efforts determining industry standards. Our processing approach was tested on two LiDARs, one LiDAR operating at a 905-nm wavelength, and one operating at 1550 nm. Data were collected from a field test site using calibrated Lambertian targets, with 10% and 94% reflectivities, at ranges from 10 to 250 m in two lanes. The point cloud was pre-processed using a method that determined bounding boxes centric to the targets and determined the spatio-temporal clustering of the points over a set number of frames. The points were then clustered using DBSCAN, a density-based method. This clustering was used to isolate and localize statistics for performance metrics and is compared in robustness to a k-means method, which displayed a timeout error for targets with nearby scene obstruction. Results validated this processing method and the resulting capability to compute metrics for all, targets with points returned for each tested LiDAR. Performance metrics, including probabilities of true and false positive detection, revisit rate, angular accuracy and precision, range accuracy and precision, cross-range resolution, and target point density, were computed. These results show that a combination of cluster processing and point-based metrics may be optimal for describing the LiDAR under test.