M-STEP: Multi-Scale Temporal Information Enhancement and Propagation for Hierarchical Visual Transformer Tracking

Temporal information, as an inherent attribute of video sequences, plays a crucial role in visual object tracking. It provides implicit cues to capture dynamic changes and invariant characteristics of both the tracking target and the environment. Adequately leveraging it enables tracking models to enhance their adaptability, robustness, and tracking performance. However, existing tracking methods often suffer from redundancy in temporal features, inadequate temporal context propagation, and reliance on empirical template updates, thus leading to suboptimal utilization of such cues. To address these limitations, this paper proposes M-STEP, a hierarchical visual Transformer tracking method that incorporates the shallow feature extraction (SFE) encoder, multi-scale template enhancement (MSTE) encoder, spatiotemporal information fusion (SIF) encoder, and temporal propagation enhancement (TPE) module into a unified framework to effectively enhance and propagate the multi-scale template and spatiotemporal information. Specifically, the SFE encoder employs a simple yet efficient MLP block to effectively preserve shallow spatial details while maintaining linear computational complexity. Second, the MSTE encoder utilizes a multi-scale pyramid pooling mechanism to capture the invariant target features under complex appearance and environmental changes, thereby maintaining the target-aware discriminative ability. Then, the TPE module combines Mamba and attention operations to fuse multi-scale temporal features, aiming to adaptively strengthen temporal correlations, reduce temporal redundancy, and ensure efficient propagation of temporal information. Finally, the SIF encoder integrates multi-scale spatiotemporal information to further improve the model's ability to capture motion and appearance diversity. Experimental results on eight benchmark datasets demonstrate that M-STEP achieves state-of-the-art performance, highlighting the critical role of multi-scale template aggregation and spatiotemporal information propagation in advancing tracking performance.