RGB-D sensor-based neural implicit semantic simultaneous localization and mapping with cross-attention feature fusion for real-time 3D scene understanding

Real-time three-dimensional scene understanding is critical for red-green-blue-depth (RGB-D) applications. However, existing simultaneous localization and mapping (SLAM) systems struggle in complex scenes due to inadequate fusion of geometric, appearance, and semantic data. We propose a neural implicit semantic SLAM system featuring a synergistic cross-attention fusion mechanism. By modulating appearance and semantics using spatial geometric priors, our method significantly improves tracking robustness against deeply degraded inputs, such as extreme lighting changes or dynamic occlusions. In addition, a hierarchical representation combined with a lightweight decoder enables multi-granularity scene understanding and high-fidelity multimodal outputs. Comprehensive experiments on the Replica and ScanNet datasets validate our design. Specifically, our system improves tracking accuracy by 32.7% compared with efficient dense simultaneous localization and mapping based on hybrid representation of signed distance fields (ESLAM) and 29.7% compared with RGB-DSLAM, demonstrating our performance gains over current state-of-the-art baselines. Furthermore, it achieves a 5% increase in reconstruction fidelity and a 10% enhancement in semantic segmentation quality all while maintaining a real-time processing rate of seven frames per second (FPS). We provide a highly robust, efficient solution for next-generation spatial mapping.