Trajectory-Centric Transformer for Multiple Objects Tracking in Satellite Videos

Multiple object tracking in satellite videos (SVMOT) is a challenging task due to the extremely small and dim target characteristics. Current learning-based tracking methods typically operate as open-loop systems, lacking feedback from tracking outputs, which makes it difficult to recover objects missed by the detector. In this paper, we demonstrate that target trajectories are crucial for target initialization, state updates, and recovery. Based on this insight, we propose a Transformer-based SVMOT method called Trajectory-Centric Transformer (TCT). The TCT comprises three trajectory oriented modules: a trajectory-prior incorporated detector, a 3D Trajectory-Deformable Transformer (TDT) encoder, and a Trajectory-Association Transformer (TAT) decoder. To recover missed targets caused by an unstable detector, we first incorporate historical information into the Sparse-UNet detector by combining target trajectories and segmented foreground as point clouds. Next, we introduce multiple trajectory hypotheses by associating targets with predicted tracking boxes and current-frame detection boxes. Unlike traditional trajectory-box association methods, we perform trajectory-level association using the Trajectory-Association Transformer. The 3D TDT encoder extracts the spatio-temporal target representation from a input video clip, and then the TAT decoder decodes the relationships among all hypotheses to update the true target state. Extensive experiments on the satellite MOT benchmark VISO demonstrate the effectiveness of our method.