Robotic system to capture images corresponding to inspector's gaze point feature for remote visual inspection of train bogies

Robotic systems for capturing images for remote visual inspections of train bogies – in a manner that reduces labor and time – are being developed. One method for intuitively determining the robot’s camera pose is to use the inspector’s head pose recorded by a head-mounted display (HMD). However, a method for extracting the camera pose from which the robot can image the inspection points from numerous head poses has not yet been investigated. Assuming that the inspector’s gaze points during inspection are highly concentrated, we propose a method that extracts the camera pose aligned with the dense clusters of gaze points collected by the HMD and a robotic system that assumes the extracted camera pose to capture images. It was experimentally validated that inspection-related gaze points are more clustered and the camera pose for capturing images can be extracted from the clustering results. The robotic system was tested in a real-world setting in which it captured images at 11 inspection points during a railway company’s periodic inspection of a train bogie. The robot-captured images were verified as usable for inspection by a human inspector, and that verification confirms the practicality of the robotic system.