Trajectory-Aware Attack: Explainable Adversarial Attack Against Multiple Object Trackers

Multi-Object Tracking (MOT) aims to build moving trajectories of objects within video sequences and serves as a critical component in autonomous driving systems. Recently, several studies have revealed the vulnerability of existing MOT methods by investigating adversarial attacks against MOT, raising significant safety concerns for real-world applications. These methods attack trackers by deliberately inserting false alarms, which mislead trajectories to drift from their correct paths. However, current MOT attack methods fail to propose efficient strategies for generating false alarms, as they either rely on computationally intensive optimization to determine the placement of false alarms, or crudely insert a large number of heuristically designed false alarms. In this paper, we propose an explainable and effective false alarm generation module, named Target Generating Module (TGM), that adaptively determines the location and size of false alarms by leveraging historical trajectory information. Based on this module, we design an attack method targeting mainstream MOT approaches, namedTrajectory-Aware Attack (TA Attack). TA Attack achieves effective disruption of MOT systems by combining detection erasure and false alarm generation, requiring only a few frames to successfully compromise trajectories. To exhibit the flexibility and effectiveness of our method, we conduct experiments using four multi-object trackers (ByteTrack, SORT, CenterTrack and FairMOT) which are enabled by two representative detectors (YOLOX and CenterNet). The results demonstrate our method achieves state of the art performance with 74.87% attack success rate on BDD100K, 81.7% attack success rate on MOT17 and 83.87% attack success rate on MOT20 while 4 frames being attacked averagely, revealing the vulnerability of association mechanism in MOT methods.