HGS-3DSeg: Identity-Encoding Half-Gaussian Splatting for Memory-Efficient 3D Reconstruction and Segmentation

Recent advancements in 3D Gaussian Splatting have achieved high-quality and real-time novel view synthesis for 3D scenes. However, this method primarily focuses on appearance and geometric modeling, lacks the ability to comprehend scenes with fine granularity at the object level. Some techniques enhance Gaussian Splatting, empowering it with the capability to perform unified 3D reconstruction and segmentation on real-world 3D scenes. Despite its improvements, these methods still face challenges in segmentation accuracy and reconstruction quality using 3D Gaussian representation. Additionally, the use of 16-bit identity codes to distinguish different Gaussian groups significantly increases memory overhead. To address these issues, we propose Identity-Encoding Half-Gaussian (ID-HGS) kernels. Our approach introduces a plane to split each Gaussian into two parts with distinct opacity values, enabling precise reconstruction of details and object boundaries. We replace the adaptive density control (ADC) used in Gaussian Grouping with localized half—gaussian point management (LHPM), which performs finer densification in under-reconstructed regions. LHPM resets pathological Gaussians and optimizes Gaussian density, reducing their impact on segmentation accuracy. Furthermore, we assign a global contribution score to each Gaussian and prune low-contribution Gaussians during training, saving memory and accelerating training. Compared to Gaussian Grouping, our method improves both reconstruction quality and segmentation accuracy while effectively controlling memory usage. Extensive experiments demonstrate that HGS-3D outperforms prior Gaussian Grouping on both reconstruction and segmentation: it achieves higher mask accuracy on the LERF-Localization benchmark and reduces peak memory usage while improving render quality on the mipnerf360 dataset.