Research on real-time guidance-oriented vision tracking and motion prediction algorithm of sports training robot

Traditional sports training methods are limited by the coach's personal experience and time and space constraints, and it is difficult to achieve efficient and personalized training guidance. In order to solve this problem, this paper puts forward an algorithm framework of sports training robot based on visual perception and decision-making ability, aiming at achieving high-robustness target tracking, low-delay motion prediction and real-time training guidance. In the visual tracking module, this paper uses HRNet (High-Resolution Network) to extract key points of human body, combines lightweight PWC-Net to extract dense optical flow field and MobileNetV3 to extract apparent features of targets, and designs an adaptive weight generation module to dynamically allocate the contribution weights of three types of features to improve the tracking stability in complex scenes. At the same time, the occlusion awareness update strategy and online template library mechanism are proposed to further enhance the tracking robustness. The motion prediction module adopts the spatio-temporal Transformer architecture, and encodes the tracking results of continuous frames into spatio-temporal sequences. Biomechanical dependence and short-term motion patterns are modeled through spatial self-attention layer and temporal convolution attention layer, and motion rationality constraints are introduced to optimize the accuracy and practicability of the prediction results. In addition, the model is lightened by knowledge distillation method, and the system is deployed on the edge equipment by combining with hardware collaborative optimization. Finally, the real-time training guidance within 20ms in a single frame is realized by multi-modal feedback. In the experimental part, in the experimental environment equipped with Jetson Xavier NX edge computing module and Intel RealSense D455 depth camera, the user-defined data set and PoseTrack2018 public data set are used for verification. The results show that this method shows high tracking accuracy and prediction accuracy in table tennis and gymnastics training scenes, and the total system delay is reduced to 25ms, which meets the real-time requirements. User feedback has a high degree of satisfaction, which verifies the effectiveness and practicability of the system.