Motion Generation for Surgical Robots using Task-Aware Attention based on Deep Predictive Learning *

In this study, we propose a method for autonomously operating a surgical robot by controlling visual attention using the robot’s own state in deep predictive learning. The proposed method, named TAIRNN (Task-Attentive Informed Recurrent Neural Network) uses a state-conditioned Query to retrieve visual Keys to deep predictive model. Experimental results of point-to-point movement showed that this method can reach the final target with improved accuracy and in fewer steps compared to conventional methods that rely solely on image information. The results demonstrate that an approach that incorporates a robot’s self-state awareness into its visual attention mechanism is effective in suppressing task-irrelevant visual noise and improving control stability.