SiamTrackCaps: A Siamese-Based Single Object Tracker Using Capsule Networks

Single object tracking (SOT) methods are commonly based on Siamese networks to extract features. Unfortunately, such features may lack important attributes like the target object's position and size. This inherently hinders tracking performance when the target object is heavily occluded or its viewpoint changes. CapsNets have demonstrated strong performance in identifying the existence of objects in images by encoding their key attributes, such as size and viewpoint. In this paper, we propose SiamTrackCaps, a novel SOT method that uses a CapsNet-based classification head in conjunction with a regression head and a centerness head. We also propose an object-frame (OF) routingby-agreement algorithm to find similarities, or agreement, between the capsules representing the target object and those representing the search frames. Evaluations of several benchmark datasets show that SiamTrackCaps outperforms existing SOT methods, including transformer-based trackers. Moreover, compared to the baseline, SiamTrackCaps achieves significant improvements even when trained with limited data.