Unifying Modality and Scale: Visual Mamba for Feature Fusion in RGB-X Crowd Counting

Crowd counting has long been a crucial topic in the domains of computer vision and video surveillance. In particular, with the widespread use of thermal or depth cameras, RGB-X crowd counting has emerged as a prominent research focus. Although depth or infrared images provide complementary information, the core challenge remains in effectively unifying heterogeneous cross-modality and cross-scale information to form a comprehensive representation of crowd distributions in complex scenes. To address this problem, we propose a novel Mamba-based framework, termed UMS-VMamba-CC for multi-modal (RGB-X) crowd counting. Specifically, we design the cross-modality disentanglement fusion visual Mamba (CMDF-VMamba) that uses self-supervised learning to decompose modality-invariant and modality-specific features in spatial-frequency domains, followed by multi-modal feature aggregation via the gating mechanism. For cross-scale fusion, we design the cross-scale pyramid fusion visual Mamba (CSPF-VMamba), which adopts the bi-directional pyramid structure to incorporate low-level features into high-level representations during downsampling, while upsampling high-level features and aggregating them with low-level features through state space contextual modeling. Comprehensive experiments on multiple mainstream datasets demonstrate that the UMS-VMamba-CC framework achieves competitive performance for RGB-X crowd counting.