SignSpotter: Scene Text-Aware Cross-Modal Semantic Consistency Verification for Street View Images and POI Data

The semantic consistency between street view images and Points of Interest (POI) is an important yet insufficiently explored issue in the quality assessment of Volunteered Geographic Information (VGI) and location-based services (LBS). In real-world mapping platforms, inconsistencies commonly arise from outdated POI attributes, visual occlusions, scale variations, and ambiguous textual cues in complex street scenes, posing challenges to reliable geographic entity validation. This paper presents SignSpotter, a multi-scale vision-language framework for entity-level semantic consistency verification between street view images and POI data. A scene-text-informed visual pre-training strategy is first introduced to enhance a Vision Transformer’s sensitivity to textual regions using large-scale scene text data. An asymmetric dual-tower architecture is then designed to jointly model multi-scale visual features and high-level POI textual features, addressing scale heterogeneity. Furthermore, a cross-modal semantic reasoning module establishes fine-grained associations between visual patches and textual tokens through bidirectional interactions, enabling explicit entity-level consistency inference. Experiments on a large-scale multi-city street view images and POI dataset demonstrate that SignSpotter outperforms representative baseline methods in terms of Precision, Recall, and F1-score. Cross-city transfer and occlusion robustness analyses further confirm the generalization ability of the proposed approach, highlighting its potential for geographic data quality control and large-scale map maintenance.