Toward Oriented Multi-Object Tracking for Fisheye Images: Dataset and Framework

Multi-object tracking (MOT) in fisheye images becomes particularly challenging due to significant radial distortion. In MOT, Camera Motion Compensation (CMC) is crucial for mitigating inter-frame errors caused by camera movement. However, conventional CMC methods degrade in fisheye images, as their rigid motion models cannot characterize the non-uniform motion fields introduced by severe distortion. In this paper, we first establish this limitation through theoretical derivation and then propose a plug-and-play ROI-Centric Motion Compensation (RCMC) mechanism, which serves as a new CMC solution that leverages instance-aware ROI for motion estimation. Moreover, fisheye distortion introduces significant target tilt, distorting object geometry and thereby hindering accurate detection and separation of adjacent instances. To address this issue, we integrate RCMC with tailored adjustments for oriented bounding boxes (OBB) to form FO-MOT, a MOT framework for fisheye images. In addition, to overcome the lack of dedicated evaluation benchmarks, we present FisheyeMOT, a comprehensively annotated dataset with OBB-based labels, designed to support the development and validation of fisheye MOT systems. Finally, we demonstrate the superiority of our framework by qualitative and quantitative experiments. The dataset and code will be released at https://github.com/lukanightfever/FO-MOT.