Deep reinforcement learning-based design with observation buffer of rolling motion for snake-like robots

Snake-like robots can travel over environments that are difficult for wheeled mobility mechanisms. However, undulating locomotion requires high power consumption. We propose an efficient method that integrates the center-of-gravity (COG) shifting for the navigation of the robot to address the aforementioned problem. In the proposed method, the snake-like robot transforms into a tire-like shape to realize a parallel two-wheeled configuration. Subsequently, by deforming the head or tail sections, the position of the COG is changed, and the resulting gravitational torque generates a rolling motion. The proposed method allows the use of rolling motion with high traveling efficiency on level ground and undulating locomotion in water as well as other uneven surfaces. The rolling motion designed in previous research was achieved by the feedback of the direction of gravity measured by an acceleration sensor. Therefore, it was only designed to be capable of traveling on smooth floors or asphalt, making it difficult to maintain straight traveling when road conditions change. This paper presents a controller design method using deep reinforcement learning (RL) to achieve robust traveling by the rolling motion. We conducted experiments using the controller designed by RL and compared the experimental results with numerical simulations. Experiments demonstrated that the RL-designed rolling motion achieved higher straightness than that of previous methods and higher traveling efficiency than conventional undulating locomotion.