Enhanced Automated Penetration Testing Using Double Deep Q-Learning

Abstract As cyberattacks grow in complexity, traditional manual penetration testing becomes increasingly time-consuming, costly, and dependent on expert knowledge. In this paper, we present an automated penetration testing framework based on Double Deep Q-Learning (DDQN) to enhance attack planning efficiency, stability, and decision-making. The framework builds realistic logical network topologies using real-world vulnerability and host data gathered from the Shodan search engine and the National Vulnerability Database. It produces attack graphs and effective attack paths using MulVAL and then subsequently transforms them into matrix representations appropriate for reinforcement learning. After comparison to the baseline Deep Q-Network (DQN), experimental results on static logical topologies demonstrate that DDQN achieves more stable learning and lower variance, with an average success rate of approximately 65% in reaching the target system. Using these results, we show how well DDQN directs ethical hackers toward effective attack tactics and illustrates the framework's potential for automated penetration testing systems and cybersecurity training.