TSTrans: Temporal-Sequence-Driven Transformer for Single Object Tracking in Satellite Videos

Satellite videos capture broad field of view of city-scale scenes, while moving objects of interest could only claim to be tiny, occupying only several pixels. Besides, noises from target-similar surroundings and illumination change exist everywhere in the complex satellite imaging backgrounds, where objects are easily occluded by man-made buildings or clouds. Therefore, considering the limited information capacity of spatially appearance and semantic features, satellite video object tracking algorithms should make the most of temporal sequence information, to cope with the above challenges involving object detail deficiency, object occlusion and environmental noises. In this paper, we focus on the critical characteristic “temporal sequence” and fully exploit its implied target changing tendency and motion continuity to boost tracker performance. The proposed Temporal-Sequence-Driven Transformer tracker (TSTrans) utilizes Transformer framework to take advantage of its long-term relation modeling ability. To obtain robust object understanding, TSTrans introduces multiple temporal templates to extract target variation pattern, thereby achieving dense target-aware attention between searching patches and long-term template sequence. For global refinement of tracking result, TSTrans resists occlusion and model attention distraction challenges through trajectory prediction based on historical frame sequence. Comprehensive experiments have been conducted on three large-scale satellite video datasets including SatSOT, SV248S and VISO. Contrasted with 12 advanced trackers, our TSTrans demonstrates state-of-the-art (SOTA) tracking accuracy, with precision and success scores of 71.00%/53.20%, 86.36%/40.84% and 65.38%/28.05% on SatSOT, SV248S and VISO respectively, running at around 50 frames per second (FPS). Code is at https://github.com/yitongli123/TSTrans.