Using the Reinforcement Learning Agent to Test the Correctness of Rule Configuration in Web Application Firewalls

Data protection is one of the most essential elements of cyber security strategies and processes. One of the components that support this process are firewalls, particularly web application firewalls (WAFs). These tools are responsible for detecting and blocking potential attacks coming from user traffic. Their proper security configuration (usage of appropriate blocking rules) is behind this crucial task, which is often marginal. One method of verifying such a configuration is using fuzz tests, but they have some drawbacks. In this work, we investigated the effectiveness of WAF configurations in the context of cross site scripting attacks (XSS) with an agent using a reinforcement learning (RL) approach.For this purpose, we created a suitable test environment with six different types of WAFs (e.g., ModSecurity, Naxsi, and SafeLine) and we conducted several experiments using the RL agent to check whether it is possible to bypass attack detection by the application firewall. Moreover, we compared the effectiveness of the RL agent with five web application fuzzers to verify whether it is capable of finding more effective payloads that can bypass WAF detection. The results allow us to conclude that using the RL concept and fuzzing can be an appropriate method for verifying the efficacy of WAF configurations.