KyVul, an LLM Created C/C++ Vulnerability Dataset

Software vulnerabilities have been the root cause of many cyber-attacks. While artificial intelligence shows promise to identify these vulnerabilities, the lack of high-quality data has hampered research in this area. Existing datasets of vulnerable code are constructed by labeling human-written code. We present KyVul, the first large language model (LLM)-generated dataset of C/C++ vulnerabilities. Our dataset’s labeling accuracy is 97%, surpassing that of most existing vulnerability datasets. Combining KyVul with PrimeVul to train a machine-learning classifier yielded an average increase of 47.5% for the area under the precision-recall curve (AUPRC) and an average increase of 1.2% for the area under the receiver operating characteristic curve (AUROC). An unpaired t-test confirmed both of these improvements to be statistically significant. This dataset demonstrates that LLMs can be used to generate large quantities of vulnerable and non-vulnerable code samples.The dataset and code used to generate and test it are at the following GitHub link: https://github.com/brysonBrown/KyVul