Autonomous Robot‐Assisted Video Capture in Open Cardiac Surgery: Surgical Field Detection and Obstacle‐Aware Viewpoint Planning via CHELS

BACKGROUND: Traditional surgical video recording requires professional personnel to manually adjust the camera capturing position, which leads to low recording efficiency and interference with the surgical process. METHOD: To address these issues, this study proposes an automatic surgical video capture scheme based on an intelligent surgical video capture robot system. The method utilises the YOLO v5 algorithm and a 3D camera to identify and localise the surgical field area. Additionally, Convex Hull Edge Line Search (CHELS) algorithm is constructed to rapidly solve the optimal capturing position while avoiding obstacles. Finally, Position-Based Visual Servoing (PBVS) is applied to control the camera for precise capture of the surgical field. RESULT: The detection model for the surgical field achieved an mAP@0.5 of 0.962, and the average solution time of the CHELS is 0.1469s. CONCLUSION: Automatic surgical video capture schemes can efficiently enable the intelligent and automatic recording of surgical videos.