Assessing cyberworthiness of complex system capabilities using the cyber evaluation and management toolkit (CEMT)

Cyberworthiness, as it is termed in Australian Defence, is a necessary feature of modern complex systems that are required to operate in a hostile cyber environment. To evaluate the cyberworthiness of complex systems or system-of-systems, the researchers developed a Cyber Evaluation and Management Toolkit (CEMT) which leverages model-based systems engineering techniques to examine a cyber-physical system’s vulnerability to, and risk of, cyber-attacks. This assessment method addresses the cyber-attack surface and threat kill chains, including supply chains and supporting infrastructure. Our previous research developing the CEMT is summarised, and the research results of a major implementation trial on a naval capability are presented. CEMT is novel to other similar assessment approaches articulated in the literature research by being threat-led and using visualisations tailored to inform decision-makers without specific cyber expertise. The research trial found strong support among all categories of stakeholders that CEMT improved the rigour in determining complex system vulnerabilities, ranking the likely cyber risks, and proposing the most appropriate mitigations to reinforce and pursue as applicable. 62.5% of trial respondents gave the CEMT a qualitative rating of Excellent or Very Good , and the recommendation likelihood mean across all respondents was 8.6 out of 10. Moreover, the assessment approach left a sound basis to maintain an informed accreditation against future cyber-threats and system changes. This large and lengthy case study on a major Defence platform provides a clear indication of suitability for further trials of the CEMT when evaluating and assessing the cyberworthiness of complex cyber-physical systems.