Skeleton integrity: A method for the efficient fine-tuning of pose estimation models for pigs

Pose estimation is a popular computer vision method for the automated analysis of animals in observational studies. Since pose estimation is a challenging task, the use of pre-trained models is widespread. However, if the shift between the domains of pre-training and application is too large, fine-tuning of the models is necessary. Pose estimation is based on keypoints, the annotation of which is costly in terms of time and effort. In most realistic settings, the available resources for the annotation of video frames are limited. Therefore, it is crucial to maximise the utility of a small number of labelled frames. This study proposes skeleton integrity, a method for selecting the frames with the highest utility for fine-tuning a pose estimation model. It works by analysing the keypoint structure of the pre-trained model's predictions and only requires the initial preparation of a single labelled frame. The method was evaluated in the context of a study on social behaviour in pigs. Skeleton integrity was used to extract a selection of 100 high-utility frames (895 pig instances) from a large dataset recorded in the study. A detailed analysis was performed, showing that frame utility is determined by variations in keypoint visibility, crowding and the resolution of the pigs. An empirical study showed that a ViTPose model fine-tuned on a skeleton integrity-based selection outperformed the same model fine-tuned on a random selection by at least 2.51% in average precision and 3.48% in average recall, underlining the importance of targeted data selection for low-data fine-tuning.