3D Building Model Segmentation using GNN and ViT
Abstract
Abstract. Reliable semantics in 3D building models support practical urban tasks such as planning, asset inventory, and maintenance. This paper presents an approach that pairs graph-based geometry (GNN) with image-based appearance (ViT) to improve component segmentation. A Graph Neural Network (GNN) is first applied to the building mesh to capture structural cues and produce initial labels. Multi-view 2D projections (orthographic and perspective) are then rendered and processed with a Vision Transformer (ViT) to recover visual patterns related to windows, doors, roofs, and walls. The two streams are reconciled through a simple consensus fusion that projects ViT predictions back onto the 3D geometry and refines the labels. In experiments, the proposed pipeline improves accuracy and classwise consistency over a GNN baseline, with clearer gains on small or visually ambiguous elements.
Fields of Study & Subjects
3D Shape Modeling and Analysis
Advanced Neural Network Applications
Remote Sensing and LiDAR Applications
Publication details
- Journal / source
- The international archives of the photogrammetry, remote sensing and spatial information sciences/International archives of the photogrammetry, remote sensing and spatial information sciences
- Publisher
- Copernicus GmbH
- Volume
- XLVIII-4/W17-2025
- Pages
- 279–283
- Publication date
- 15 January 2026
- Language
- EN