Intelligent Robotic Hand with Real-Time Visual Feedback

This research work investigates the inclusion of machine learning methods into a robotic hand designed mainly for tasks like cleaning, object sorting, sample collection, and experimental support. By using a convolutional neural network to read and react to visual inputs, the robotic hand can execute complex manipulating tasks with great accuracy and efficiency. Basic components of the system include cameras for visual perception, actuators for physical manipulation, and machine learning algorithm for motion interpretation and object recognition. The proposed design uses MediaPipe and OpenCV to create a real-time object tracking and recognition system. To identify object landmarks and verify object positions inside an established region of focus, the camera's visual input is processed. For later control actions, the collected digital object states are sent via serial communication to a Raspberry pi board. According to experimental results, the proposed approach reduces processing time on average by about 15% when compared to other methods.