Digital twin-assisted automatic ship size measurement for ship–bridge collision early warning systems

Long-span bridges are usually constructed over waterways that involve substantial ship traffic, resulting in a risk of collisions between the bridge girders and over-height ships. The consequences of this can be severe structural damage or even collapse. Accurate measurement of ship dimensions is an effective way to monitor approaching over-height ships and avoid collisions. However, the performance of current techniques for estimating the size of moving objects can be undermined by large sensor-to-object distance, limiting their applicability. In this study, we propose a digital twin-assisted ship size measurement framework that can overcome such limitations through a predictive model and virtual-to-real-world transfer learning. Specifically, a 3D synthetic environment is first established to generate a synthetic dataset, which includes ship images, positions, and dimensions. Then the pixel information and spatial coordinates of ships are adopted as regressors, and ship dimensions are selected as the output variables to pre-train deep learning models using the generated dataset. Coordinate system transformations are applied to address dataset bias between the simulated world and real-world, as well as improve the model’s generalization. The pre-trained models are compared using supervised virtual-to-real-world transfer learning to select the version with optimal real-world performance. The mean absolute percentage error is only 3.74% across varying camera-to-ship distances, which demonstrates that the proposed method is effective for over-limit ship monitoring.