A Computer Vision Approach for Performance Tracking of Robotic Compliant Systems

ABSTRACT Characterization and testing of 3D‐printed robotic compliant systems for lifespan assessment is time‐consuming and costly. For this reason, this work introduces a computer vision approach for automated, non‐invasive monitoring of grippers and evaluation of failures. The vision system first detects colored fiducial markers placed on key points of the gripper. The detection model was trained using synthetic data to ensure robustness to background, illumination, and gripper color variations. Then, the marker positions across frames are used to train and detect anomalies in the gripper's displacement. This is performed by thresholding the reconstructed signal over temporal analysis windows, using the reconstruction error as an anomaly score. Validation was performed on real 3D‐printed grippers under controlled mechanical failures and uncontrolled lighting and background conditions, correctly classifying over 97% of actions corresponding to normal and anomalous gripper performance. The proposed framework offers a scalable and low‐cost alternative to embedded sensors for monitoring gripper performance and detecting early failures.