Beyond Fitness: Revolutionary Exercise Tracker Combining Pose Recognition with Heart Rate Monitoring using Remote Photoplethysmography (rPPG)

Heart rate (HR) is a critical indicator in fitness monitoring, athletic performance evaluation, and injury prevention. However, traditional motion-sensitive wearable devices are highly susceptible to movement artifacts, which degrade measurement accuracy during physical activity. Remote photoplethysmography (rPPG) offers a non-contact alternative for HR measurement, though it too remains sensitive to motion. This study proposes a novel framework that integrates MediaPipe’s BlazePose for bicep-curl motion tracking with rPPG-based real-time HR monitoring. The system leverages pose-estimation data to accurately track arm movements and applies advanced signal-processing techniques to suppress motion-induced noise in the rPPG signal, thereby enhancing HR estimation accuracy. The extracted signal is pre-processed using filtered, normalized, and transformed from the time domain to the frequency domain, enabling reliable HR extraction during continuous exercise. The green color channel is selected for BPM estimation, as it exhibits a strong correlation with cardiac pulse signals. Experimental results demonstrate that motion-corrected rPPG can effectively support real-time, non-contact HR monitoring during exercise. This approach shows strong potential as a platform for personalized fitness coaching and AI-based workout optimization. Future work will focus on extending the system to support multi-exercise tracking and continuous performance monitoring. The proposed framework enhances user safety by enabling early detection of abnormal heart rate patterns during exercise, such as tachycardia or irregular fluctuations caused by overexertion. By correlating physiological signals with biomechanical movement data, the system can adapt workout intensity in real time, ensuring exercises remain within safe cardiovascular limits. This closed-loop feedback mechanism supports individualized training plans tailored to the user’s fitness level and physiological response.