Conceptional Guidelines for Robustness Evaluation of Autonomous Agents

Autonomous vehicles rely on high-quality sensor data, such as from camera and Light Detection and Ranging (LiDAR) for their driving decisions. Object detection and motion planning are core tasks in autonomous driving, both requiring robustness against diverse sensor perturbations. While robustness evaluation for object detection is well studied, far fewer approaches exist for end-to-end (E2E) motion planning agents. This paper outlines key differences between object detection and E2E motion planning and discusses resulting challenges for robustness evaluation. We present an empirical study comparing the robustness of the object detection system LoGoNet and the E2E agent TransFuser++, both using camera and LiDAR inputs, under LiDAR-only perturbations. Based on our findings, we propose requirements and a parameterization scheme for perturbations to guide the design of future robustness evaluation frameworks for autonomous agents.