Radar and Event Camera Fusion for Agile Robot Ego-Motion Estimation

Achieving reliable ego motion estimation for agile robots, e.g., aerobatic aircraft, remains challenging because most robot sensors fail to respond promptly and clearly to highly dynamic robot motions, often resulting in measurement blurring, distortion, and delays. In this article, we propose an inertial measurement unit (IMU)-free and feature-association-free framework to achieve aggressive ego-motion velocity estimation of a robot platform in highly dynamic scenarios by combining two types of exteroceptive sensors, an event camera and a millimeter-wave radar. First, we propose instantaneous raw events and Doppler measurements to derive rotational and translational velocities directly. Without a sophisticated association process between measurement frames, the proposed method is more robust in textureless and structureless environments and is more computationally efficient for edge computing devices. Then, in the back-end, we propose a continuous-time state-space model to fuse the hybrid time-based and event-based measurements to estimate the ego-motion velocity in a fixed-lag smoother fashion. In the end, we validate our velometer framework extensively in self-collected experimental datasets featured by aggressive motion and high dynamic range (HDR) lighting conditions. The results indicate that our IMU-free and association-free ego motion estimation framework can achieve reliable and efficient velocity output in challenging environments.