A Meta-Perspective on Usable Security Research Through Advice and Transparency

Usable security research emerged about 30 years ago and is now an established sub-discipline of security and privacy (SP) research. The field made many contributions in that time, but many challenges remain in real-world applications and for practitioners. In this thesis, I reflect on the current state of usable privacy and security (UPS) and its impact from two viewpoints: how we, as researchers, impact practice through advice, and how we, as researchers, impact our research through research reporting transparency. I contribute four publications to this cumulative dissertation. The first paper investigates usable security advice from online sources for practitioners and uncovers many shortcomings, such as an overload of scattered pieces of advice that might contradict each other or are outdated. If software professionals receive only low-quality and outdated advice, it is not surprising that real-world applications often have poor usable security. In the second paper, I investigate artificial intelligence (AI) assistants as a novel source of advice but find many parallels with other online advice sources. Practitioners frequently use AI assistants without trusting the AI assistants’ security capabilities, but try to scrutinize AI suggestions. Hence, improving AI assistants and their security capabilities should be a major concern. My third and fourth publications are meta-research in which I investigate transparency in UPS research through interviews and a literature analysis. The results show that researchers appreciate transparency, but current drawbacks and challenges prevent larger efforts because these outweigh incentives. Moreover, there is an implicit community standard that leads to inconsistencies in research reporting and generally leaves room for improving transparency. Transparency is necessary to evaluate publications (e.g., as reviewers) and ensure that other researchers can build on existing research or replicate it. Based on my insights, I provide recommendations on improving the advice ecosystem and reporting transparency in UPS research. Both can increase the impact of research; on practice through transferring knowledge and advice from academia, and on its own advancement through transparency that facilitates replications, meta-research, and building upon others’ research. Overall, the UPS community should further pursue meta-research to advance itself by critically reflecting and learning from the past.