Automated Structural Inspection with Mask2Former and VGGT for Drone-Assisted Surface Defect Segmentation and 3D Visualization

Abstract Canada’s Core Public Infrastructure Survey reported that nearly 40% of rural highways and bridges were in fair condition, with over 12% rated poor or very poor. These statistics underscore the urgent need for proactive, data-driven inspection methods to prevent failures and protect public safety. In response, this study presents an automated structural inspection framework that leverages drone imaging and deep learning to enhance defect detection and visualization. High-resolution images of bridge pier surfaces are collected using drone and subsequently processed with the Masked-attention Mask Transformer (Mask2Former), which has been trained on publicly available datasets. Pixel-level segmentation of defects, specifically cracks, is achieved. To place these results into geometric context, the Visual Geometry Grounded Transformer (VGGT) is applied to reconstruct a 3D representation of the structure. The segmented cracks are integrated directly into the reconstructed geometry, giving engineers an intuitive and spatially aware 3D view of structural conditions. This combination enables both comprehensive surface analysis and clear spatial localization of damage. Field validation is carried out on the Low Level Bridge in Edmonton, Alberta. Results demonstrate that the proposed framework can operate reliably, providing a scalable and efficient tool for condition assessment and infrastructure management.