Retention vision transformer: a hidden state-driven efficient transformer for driver behavior recognition

Convolutional Neural Networks (CNNs) have played a pivotal role in driver behavior recognition but often neglect global feature dependencies when stacking layers. Although transformers alleviate this issue, their high training costs hinder practical deployment. To address this challenge, we propose the Retention Vision Transformer (RVT), a lightweight architecture designed to reduce training overhead. Built upon the stage-wise design of TWINS, RVT incorporates a retention mechanism with Rotational Position Encoding (RPE), enabling parallel attention computation through a hidden state matrix. We evaluate RVT on the publicly available State Farm dataset, comparing it with several representative vision transformers and the lightweight CNN MobileNetV3. RVT achieves over 99% accuracy with only 10.1 M parameters and 0.7GB memory usage, reducing training memory consumption by approximately 80%, while exhibiting faster convergence. Although each training step incurs slightly higher computational cost than MobileNetV3, RVT converges within about 1/10 of the training epochs, resulting in lower overall computational requirements. These results demonstrate that RVT effectively reduces image size requirements, parameter count, computational overhead, and memory usage, while maintaining state-of-the-art accuracy. The proposed method thus holds strong potential for real-time driver behavior recognition and related applications.