An Efficient Internet-Wide Port Scan Algorithm Based on Deep Reinforcement Learning

A critical concern regarding the security vulnerabilities of Internet of Things (IoT) devices has been repeatedly highlighted in existing research. Considering resource limitations, the Internet-wide port scan (IWPS), a well-established vulnerability scan scheme, has recently gained attention for its applicability to IoT networks. There is an urgent need to develop Internet-wide scanning solutions that can achieve both high scanning rates and high reachability. In this paper, we focus on open scans and propose an Area-aware IWPS algorithm based on deep reinforcement learning (DRL). We first construct an average delay table based on the physical locations of the port scanners and the targets. To solve the problem efficiently, we formulate the problem as a Markov Decision Process (MDP) whose reward function is designed based on the average delay table. Then, a DRL-based algorithm is proposed to achieve efficient port scanning. Finally, we conducted a large number of experiments to verify the efficiency and reachability of the algorithm. Compared with the most popular open scan tool, Nmap, the scan rate of our proposed policy is 4–5 times faster, and the detection reachability increased by 6%.