A Double-Layer Autonomous Decision-Making Method Based on Expert Knowledge and Deep Reinforcement Learning

The underwater environment is complex and volatile, where underwater unmanned systems face the dual challenges of incomplete perceptual information and environmental uncertainty. Traditional decision-making methods highly rely on complete perceptual data and map information, resulting in insufficient adaptability in dynamically complex scenarios and difficulty in efficiently completing tasks such as autonomous navigation and obstacle avoidance. To address the above challenges, this paper proposed a double-layer autonomous decision-making method based on expert knowledge and deep reinforcement learning, aiming to enhance the adaptive capacity of unmanned systems in underwater intelligent decision-making and significantly improve the efficiency of task execution. Specifically, a double-layer autonomous decision-making architecture consisting of seven functional modules was first designed to effectively ensure navigation safety by strengthening system robustness. Secondly, an autonomous decision-making strategy generation method integrating expert knowledge and deep reinforcement learning was proposed to improve the adaptability of underwater unmanned systems in unknown scenarios. Finally, a multi-module design method was proposed to achieve the decoupling of each functional module, effectively improving the research and development efficiency of unmanned undersea systems. By taking unmanned undersea systems as the research object, experiments on autonomous navigation and obstacle avoidance were conducted on the Unity virtual simulation platform. The results show that the success rate and the convergence speed of the average reward value of the proposed method are superior to those of benchmark methods such as proximal policy optimization and soft actor-critic, providing solid theoretical support for autonomous decision-making in real-world scenarios.