Real-Time Hand Pose Estimation Using FMCW Radar on Resource-Limited Edge Devices

Real-time hand pose estimation is essential for natural human-computer interaction (HCI), especially in situations where vision-based systems face challenges due to lighting, occlusion, or privacy issues. This paper introduces RadarPose, an innovative framework that estimates 2D hand keypoints for HCI applications using raw Frequency-Modulated Continuous Wave (FMCW) radar signals. To address the difficulty of manually annotating FMCW radar data, a cross-modal supervision strategy is implemented, utilizing synchronized RGB video frames processed by a pre-trained vision model to generate hand keypoint labels for training the radar signals. RadarNet, the core of this framework, is a lightweight convolutional neural network (L-CNN) designed to predict 21 hand keypoints, handedness, and hand presence directly from a raw data cube. By removing the Fast Fourier Transform (FFT) stages, this framework facilitates end-to-end learning in the radar complex data domain, resulting in significantly reduced latency. Once trained, the proposed framework relies solely on radar data for inference, ensuring both privacy protection and invariance to lighting conditions. The model comprises only 326k parameters and achieves real-time inference in just 1.19 milliseconds on an NVIDIA Jetson Nano. Experimental results indicate that RadarPose demonstrates competitive accuracy compared to vision-based models in well-lit environments and significantly outperforms them in low-light or occluded scenarios, highlighting its potential for robust, edge-compatible hand pose estimation. The code and dataset were available at https://github.com/thetuantrinh/UWB-Radar-Hand-Pose-Estimation.git.