PCAP-SL: Pedestrian Crossing Action Prediction Based on Sequential Learning

Predicting and understanding pedestrian behavior is one of the main challenges faced by autonomous vehicles in urban environments. Recent studies have shown promising results by introducing complex and large deep learning models to predict pedestrian actions. Despite being powerful, these models require significant time for both training and testing. However, when these models are embedded in vehicles, such delays become problematic for real-time inference. Additionally, capturing the spatial and temporal interactions between pedestrians and other traffic elements, such as vehicles, remains a significant challenge. To address these issues, this paper proposes an approach that applies sequential learning to traditional machine learning techniques, such as lightweight models including the XGBoost classifier, to determine whether pedestrians are crossing the road or not. Our model PCAP-SL (Pedestrian Crossing Action Prediction with Sequential Learning) uses pre-trained and frozen Vision Transformers (ViT) for feature extraction, enabling more efficient capture of complex interactions between pedestrians and their environment. The approach is validated against state-of-the-art models using the Joint Attention for Autonomous Driving (JAAD) and Pedestrian Intention Estimation (PIE) benchmark datasets. The results demonstrate that the proposed approach achieves competitive performance in predicting pedestrian actions while maintaining efficient inference time making it suitable for integration into real-time autonomous driving pipelines. The lightweight nature of the XGBoost classifier, combined with frozen ViT feature extraction, offers a practical balance between accuracy and computational efficiency.