Psyzkaller: Learning from Historical and On-the-Fly Execution Data for Smarter OS Kernel Fuzzing

Read full text
Authors: Boyu Liu, Yang Zhang, Liang Cheng, Yi Zhang, Junjie Fan, Xiaoshan Sun, Yu Vincent Fu, Zhen Li, Dengguo Feng
Cited by: 0
References: 37

Abstract

OS Kernel fuzzers such as Syzkaller often struggle to generate syscall sequences that respect intrinsic Syscall Dependency Relations (SDRs), resulting in seeds that either violate kernel constraints or fail to reach deep execution paths. We propose leveraging an N-gram model to learn SDRs from both kernel execution history and ongoing fuzzing results. This enables the fuzzer to capture dependencies in similar kernel versions while adapting to target-specific behaviors, thereby improving the validity of generated seeds. Additionally, we introduce a bidirectional Random Walk strategy to enhance the diversity of generated seeds. We implement this approach in a prototype, Psyzkaller, on top of Syzkaller. Experiments show that, trained with the large-scale DongTing dataset and continuously updated with ongoing fuzzing results, Psyzkaller improves Syzkaller's code coverage by 4.6%-7.0%, triggers 110.4%-187.2% more crashes, and discovers eight previously unknown kernel vulnerabilities. Furthermore, Psyzkaller outperforms state-of-the-art fuzzers such as ACTOR and SyzDescribe in both coverage and crashes.

Fields of Study & Subjects

Security and Verification in Computing Software Testing and Debugging Techniques Advanced Malware Detection Techniques

Publication details

Journal / source
2026 56th Annual IEEE International Conference on Dependable Systems and Networks (DSN)
Publisher
IEEE
Pages
96–108
Publication date
22 June 2026
Language
EN