Consistency Enhanced Deep Learning for Visual Perception Data of Structural Health Monitoring

The multimodal data for visual perception (such as images, point clouds, etc.) are obtained under similar environments and load conditions for the same structural system, and exhibit typical temporal, spatial, and high-dimensional feature correlations. Moreover, the visual perception data of different structures also have similarities in multi-dimensional features such as material properties, damage types, and geometric shapes. By fully leveraging the inherent correlations and similarities within the visual perception data, it could reduce reliance on the synchronicity of multi-modal data samples, as well as their annotation quantity and completeness. Therefore, establishing a weakly-supervised deep learning method with consistency constraints is a possible solution to achieve precise and efficient engineering structure state perception, damage identification, and three-dimensional modeling. This talk presents a consistency enhanced deep learning method for visual perception data, mainly including: (1) cross-modal alignment of images and point clouds with cycle consistency constraints, (2) few-shot structural damage semantic segmentation with cycle consistency constraints, (3) generic unsupervised semantic segmentation for structural surface damage guided by high-dimensional feature consistency, and (4) perspective geometry consistency embedded few-view diffusion model for three-dimensional reconstruction.