Effect of markers in training dataset for markerless applications in biomechanics

The quality of dataset annotations used to train markerless motion capture models is crucial for obtaining reliable joint center estimations from videos. Because manually annotated datasets such as COCO are unsuitable for biomechanical applications, a common recommendation is to use videos synchronized with marker-based MoCap datasets. In these systems, reflective markers are placed on the skin surface, ideally on bony landmarks, and are then tracked by optical cameras to obtain highly accurate joint center annotations. While previous studies have suggested that visible reflective markers on images could bias model training, this effect has not been formally demonstrated. To address this, we used two MoCap datasets: one with 26 subjects each equipped with reflective markers mounted to rigid bodies, and the second, with 10 subjects with markers placed on bony landmarks. This allowed us to train pose estimation networks on images having visible markers. The models were then evaluated on test images with visible and inpainted markers. Our findings showed that when models were evaluated on images with markers inpainted, pixel position errors increased by +4.7% to +51.1% versus images with visible markers. This indicates that the presence of markers in training images can affect human pose estimation algorithms. To still utilize accurate annotations from MoCap, we recommend training on images with markers removed via inpainting. We also demonstrated that, in that case, the network does not rely on inpainted areas to estimate joint centers, thus making it a viable solution to the presence of markers in training images.