WiViFormer: Crowd counting based on multimodal fusion of visual image and wireless signal

Current crowd counting models often rely on single-modal information, such as visual images or wireless signal data, leading to potential information loss and unsatisfactory recognition outcomes. To address this issue, we proposes a multimodal fusion-based crowd counting model, WiViFormer. Leveraging the strengths of Transformer networks, this model achieves fusion between Channel State Information (CSI) and visual image data. To overcome Transformer networks’ limitation of extracting only global features without capturing finer-grained characteristics, this study integrates Convolutional Neural Networks to extract local features. Moreover, owing to the computational complexity of Transformers, resulting in longer training and inference times, we introduces Linear Attention as an optimization to the original attention mechanism, reducing model complexity and enhancing training efficiency. To validate the effectiveness of the WiViFormer model, extensive experimental evaluations are conducted. The experimental results demonstrate that the model yields minimal errors while maintaining high efficiency. The code is available at https://github.com/zhecui-zc/WiViFormer_Crowd-Counting .