Learning Sequential Deception Defense Strategy Against APT Using Stackelberg Markov Game

Advanced Persistent Threats (APTs) have become one of the most prominent cybersecurity risks globally. The external network-facing (ENF) services (e.g., e-commerce platforms) within a system are particularly vulnerable, as they are directly exposed to the Internet and often serve as the primary targets for attackers. By deploying deception resources to protect these ENF services, defenders can detect threats early, block potential attacks, and enhance overall system resilience. However, most existing studies on cyber deception strategies assume simultaneous moves by both attacker and defender. Furthermore, few works have considered the evolution of the system state resulting from APT attacks on the ENF services. To address these limitations, this paper proposes a Cyber Deception Stackelberg Markov Game (CDSMG) for protecting ENF services, which dynamically captures state transitions and accurately characterizes the strategic interactions between defenders and APT attackers. In CDSMG, the defender acts as the leader, who proactively selects a subset of services to deploy the deception resources based on the current system state, while the APT attacker plays as the follower, making a best response which incorporates the defender’s policy into its own strategy. To overcome the challenge of the combinatorial optimization problem of selecting a subset of services, we propose a revised version of the PPO algorithm by using no-replacement sampling to select multiple services at once, thereby significantly reducing the action space size. Finally, experimental results demonstrate that our approach effectively defends against APT attacks. It not only outperforms several baseline methods but also exhibits better scalability and robustness under varied model parameter settings.