Multi-Stage Learning for Visually Similar Road User Detection and Tracking

This paper presents a unified deep learning framework for multi-class detection and tracking of visually similar road users in complex road environments. The proposed system is designed to accurately distinguish and track various road entities—such as cars, buses, trucks, bicycles, motorcycles, pedestrians, and riders—which often exhibit similar visual features. To achieve robust detection and instance-aware tracking, we adopt a three-stage training strategy: (1) a supervised multi-task learning stage for joint object detection and class identification, (2) a self-supervised contrastive learning stage for extracting instance-level feature embeddings without identity labels, and (3) a fine-tuning stage to improve object identification accuracy by refining both the detection and feature embedding heads. The unified network simultaneously outputs object classes, locations, and appearance embeddings, enabling consistent identity association across video frames. Experimental results demonstrate that the proposed method outperforms existing approaches in both fine-grained detection and multi-object tracking accuracy.