A hybrid transformer deep Q-network for path planning

Reinforcement learning has shown strong potential for robotic path planning in dynamic environments. However, existing methods often struggle to capture complex spatiotemporal dependencies involving moving obstacles. In this work, we propose a Hybrid Transformer Path Planner (HyTra-Planner), a Parameterized Deep Q-Network (P-DQN)-based framework designed for efficient and reliable navigation in environments with both static and dynamic obstacles. HyTra-Planner presents four key innovations. First, a History Memory Generation Network (HMGN) dynamically updates temporal memory by combining current spatial features with historical information, offering a richer representation of environmental dynamics. Second, a Hybrid Transformer consists of spatial and temporal attention modules to capture both spatial and temporal dynamic patterns of obstacles. Third, an Adaptive Step Strategy allows the agent to adjust its step size according to local risks, improving navigation efficiency compared to fixed-step methods. Finally, a novel Dense Reward Mechanism is proposed to consider goal proximity, obstacle avoidance, and action efficiency altogether, which delivers continuous and informative feedback that accelerates convergence. Extensive experiments in both dynamic and hybrid environments demonstrate that HyTra-Planner achieves higher success rates, faster convergence, and more efficient path planning than state-of-the-art DQN baselines, validating its potential for real-time robotic navigation.