Action Transition Recognition using ST-GCN for Worker Following in Agricultural Support Robots

In recent years, the increasing labor burden in Japanese agriculture hs become a serious issue, driving the development of robots to assist in transporting harvested crops. This study proposes a method that recognizes the action transitions of agricultural workers and enables smooth transport assistance by utilizing three-dimensional skeletal information obtained from RGB-D images. Specifically, we employ Spatial Temporal Graph Convolutional Networks (ST-GCN) to detect the transition from "harvesting" to "loading." The recognition results are used to control the robot so that it approaches the worker before the loading action begins. The proposed method introduces a new labeling scheme tailored to harvesting and crop-loading motions, thereby improving recognition performance with ST-GCN. Evaluation experiments verified its generalization capability to different harvesting postures and workers, demonstrating an 18.7% improvement in action transition recognition accuracy compared with conventional methods. Furthermore, in robot-following experiments with the proposed method implemented, we confirmed that the system could both recognize action transitions and adjust the target following distance before the worker started loading. These results show that an ST-GCN specialized for agricultural tasks can effectively recognize harvesting action transitions, contributing to reducing the burden on workers during crop transportation.