Evaluating the Effectiveness of CSPM Tools in Detecting Cloud Misconfigurations

Despite heavy adoption of the cloud, misconfigurations are often overlooked and remain a significant issue in cloud security. With Cloud Security Posture Management (CSPM) tools available to counter cloud misconfigurations, their effectiveness, like their existence, is often unknown, misunderstood, or underutilised. This paper helps understand free tools, including both AWS and Azure first-party CSPMs and a third-party CSPM (Prowler), and assesses the effectiveness of an AI-generated DIY misconfiguration scanner in comparison to the Azure and AWS cloud environments. These tools were evaluated against 28 misconfigurations in the Azure environment and 24 equivalent misconfigurations in the AWS environment, including VMs, storage services, databases, and cloud identity platforms. With detection rates ranging from 7.1% to 78.6%, no tool met the 80% target; however, Prowler performed best across both environments, achieving 4 out of 8 severity-specific targets. The DIY scanner held the weakest overall performance of the three, whilst the first-party scanners had held satisfactory but poor performance between the DIY scanner and the third-party CSPM. While performances varied, the experiment highlights that CSPM tools can be useful, though they should not be relied on exclusively due to gaps in detection and over-sensitivity. Finally, concluding that CSPM tools can be an aid, though they should be combined with additional tools such as XDRs to help prevent security incidents. However, opportunities for additional future research remain open, due to the budget and time constraints imposed by this experiment.