Stereo YOLO UAV Localization and Tracking Enabling Autonomous Sensor Deployment on Critical Infrastructure

Unmanned Aerial Vehicle (UAV)-based structural health monitoring (SHM) systems have gained significant traction in recent years. These systems typically collect data using onboard UAV sensors (such as cameras) or through the placement of physical sensors on the structure itself (such as strain gauges). While UAVs could be manually piloted to deploy these sensors, this approach is limited in hazardous or hard-to-reach areas. To address this, we present the use of object-tracking machine learning algorithms to support end-to-end autonomous control systems for sensor placement. The algorithm is trained on multiple images of the UAV within the flight environment to determine its relative position in three-dimensional space. These positional coordinates provide critical data on the UAV’s location, guiding it toward the desired sensing point on the structure. Our system employs two externally placed cameras, oriented 90 degrees apart, to enable a full 3D assessment of the UAV’s surroundings during flight. These cameras feed visual data to a processor, which analyzes the imagery and outputs coordinates for the UAV. These are then compared to the known coordinates of the target structure. Based on this comparison, motion commands for accurate alignment and docking can be calculated. This approach enhances the UAV’s spatial awareness during flight, enabling safer and more reliable sensor deployment. The setup also reduces the risk of collisions, facilitates operations in hazardous environments, and enables sensor placement in previously inaccessible areas. Moreover, the algorithm is adaptable and capable of tracking various objects for broader applications in sensor delivery and visual navigation. The mean squared error of the model showed no further improvement after 96 of 196 epochs of training, with a training box loss of 0.770 square pixels and a validation box loss of 1.227 square pixels. Overall model summary after training and validation consists of 218 layers and 25,840,339 parameters for the UAV. Validation errors stabilized at the end of the training process, while training errors showed continual improvements. Average model error during data collection was 5.850 pixels for X-direction movement, 3.744 pixels for Y-direction movement, and 3.534 pixels for Z-direction movement of the UAV.