LAKALMANTracker: Robust Learning-Aided Kalman Filtering for Multi-Object Tracking

Kalman filter (KF)-based multi-object tracking methods assume linear object motion; however, they fall short in accurately predicting nonlinear object motion due to model mismatch. To address this limitation, we explore data-driven motion estimators and thus propose Patching-KalmanNet (PKNet), inspired by KalmanNet, which learns the Kalman gain (KG) via deep neural networks. PKNet takes multi-step historical state patches as input and leverages two key components for KG learning: variate-independent patch embedding (capturing heterogeneous motion features across different variates) and attention mechanisms (highlighting critical historical information for feature refinement). With a KF-alike structure, it can integrate directly into existing trackers as a replacement for the KF, forming the Learning-Aided Kalman Tracker (LAKalmanTracker). Experiments on SoccerNet, DanceTrack, and other benchmarks with highly nonlinear object motion show that PKNet significantly outperforms both the KF and existing KalmanNets in boosting tracker performance and robustness.