Program-plate: a method for identifying the ability to extract vulnerability features

Abstract Although there are numerous advanced and well-established models and methods available for code auditing tasks, their interpretability remains a significant challenge. For machine learning models designed to address code auditing problems, we often know that they can identify vulnerable code but lack insight into their decision-making criteria or whether they have effectively captured the characteristics of vulnerable code. To evaluate the capability of such models in extracting vulnerability-related features, this paper proposes a method called Program-PLATE. By extending a single vulnerable file into a PLATE-dataset, this method enables a more objective assessment of the model’s performance on the PLATE-dataset. We applied this method to evaluate multiple models, conducted an in-depth analysis based on the results, and provided suggestions and expectations for future research directions.