A Transformer-Based Tracker Integrating Motion and Representation Information

The appearance information of the target has been used as the only tracking cue for most trackers to locate the target in the video. However, when the surrounding environment changes drastically or there are similar interference targets, it usually causes target drift. We propose a tracker named MRTrack, a transformer-based spatiotemporal information de coupling network architecture to enhance the target tracking capability. We designed two training schemes to explore the effective integration of motion cues derived from optical flow with representation information. The first approach integrates the target's motion and representation information for training. The second scheme is a step-by-step training, where the target motion information is first learned, and the learned model is used for representation learning. We compare the two training methods on five generic tracking datasets. The experiment results indicate that the first training approach can better integrate motion and representation information, leading to more precise tracking results for MRTrack compared to solely relying on the appearance model. In addition, optical flow cues are used only in the training phase to guide the tracker in understanding motion information, and no additional cost is incurred during tracking inference.