Simulating Brain Learning Mechanisms: A Hierarchical Self-Organizing Reinforcement Learning Framework for Robot Navigation

Brain-inspired intelligence seeks to simulate the learning mechanisms of the brain in order to design efficient computational models. This article proposes a hierarchical self-organizing reinforcement learning framework for robot navigation, aiming to address the challenges of low data utilization and poor learning efficiency in training, while enhancing navigation success rates and path optimization. Specifically, first, inspired by the brain's hierarchical processing mechanisms, a low-level policy is designed to guide the robot toward the target and a high-level policy to ensure its safety. Second, deep neural network is employed to simulate the slow-learning characteristics of the neocortex and the self-organizing map is adopted to capture the fast-learning characteristics of the hippocampus. The integration of these two systems forms a self-organizing reinforcement learning algorithm that leverages their complementary learning mechanisms to significantly improve the robot's learning efficiency and decision-making quality in navigation. Third, a multimetric prioritized sampling strategy that incorporates an experience familiarity factor is designed to modulate both the temporal-difference error and Bellman error, enabling the algorithm to adaptively adjust sampling priorities and thus enhance learning efficiency. In addition, a subgoal driven strategy is proposed to reduce unnecessary exploration during navigation, further improving sampling efficiency. Finally, experiments both in the Gazebo simulator and on the Autolabor Pro1 platform are conducted. Ablation studies validate the effectiveness of the proposed components, comparative experiments demonstrate the superiority of the proposed algorithm in terms of navigation success rate and path length, and sim-to-real experiments highlight its generalization capability and practical applicability.