MCIVA: A multi-view pedestrian detection framework with a central inverse nearest neighbor map and a view adaptive module

Multi-view pedestrian detection is an important task and has many applications in areas such as surveillance and smart cities. Despite the significant performance improvements achieved in recent multi-view pedestrian detection methods, there are still three main challenges for this task. 1) In crowded areas, neighboring connected components may merge in dense regions, resulting in unclear localization of pixel peaks for each pedestrian. 2) The loss functions used in previous multi-view pedestrian detection methods have a high response to the background. 3) The camera parameters have not been fully utilized; they are only used to generate a fixed-value projection matrix. To address these challenges, we propose a novel multi-view pedestrian detection framework with Central Inverse Nearest Neighbor map and View Adaptive Module ( MCIVA ). A Central Inverse Nearest Neighbor (CINN) map is introduced to generate the ground-truth Probability Occupancy Map (POM) based on annotations, providing more precise location information for each pedestrian. To enhance the model’s attention to local structural information, we propose a local structural similarity loss to reduce the influence of false local maximum in background regions. Moreover, a novel plug-and-pull View Adaptive Module (VAM) is introduced to utilize the camera parameters to generate learnable weights for multi-view features fusion. We evaluate the proposed method on three benchmark datasets, and the results show that the proposed MCIVA significantly improves the quality of prediction map and achieves state-of-the-art performance.