Transformer-Based Multilevel Semantic Guidance and Multibranch Feature Fusion Network for Driver Attention Prediction

Most existing driver attention prediction models rely on 2D or 3D convolutional neural networks for static or local temporal modeling, which limits their ability to capture long-range spatiotemporal dependencies across continuous frames in dynamic driving scenarios. To address this limitation, we propose a novel Transformer-Based Multi-Level Semantic Guidance and Multi-Branch Feature Fusion Network (TMSFF-Net), which is composed of a multi-level semantic-guided encoder and a multi-branch prediction decoder. In the encoder, we design a multi-level semantic guidance (MSG) pathway leveraging multi-scale features extracted from the Transformer architecture to facilitate the top-down propagation of high-level semantic information, thereby enhancing semantic flow and enabling effective resolution restoration and content reconstruction. In the decoder, a gated attention (GA) mechanism is introduced to each branch to adaptively adjust feature weights, thereby highlighting key regions and enhancing saliency representation. Multiple decoder branches are then used to generate layered saliency maps, which are further fused to produce the final attention prediction. The experimental results on the DADA-2000 and TDV datasets demonstrate that the proposed TMSFF-Net outperforms state-of-the-art methods across multiple evaluation metrics, achieving superior prediction performance.