Anonymous Artifact: CodeDevia
Summary
No summary yet. VIRAT can read this paper and write a plain-language summary of its aim, methods, key findings and limitations.
Summarise with VIRAT Free to try with your Science Catalogs account.Abstract
This artifact is submitted anonymously for peer review. It contains the source code, scripts, and processed datarequired to reproduce the experimental results in the paper. All identifying information has been removed for double-blind review. Additional Baseline Type Method F1 P R DL VulSim [1] 0.541 0.418 0.765 Fine-tuned PLM CodeBERT [2] 0.571 0.437 0.824 LLM Vul-RAG-DeepSeek [3] 0.449 0.433 0.441 Fine-tuned LLM llmxcpg [4] 0.457 0.432 0.483 Ours CodeDevia 0.621 0.513 0.785 [1] Shimmi S, Rahman A, Gadde M, et al. {VulSim}: Leveraging similarity of {Multi-Dimensional} neighbor embeddings for vulnerability detection[C]//33rd USENIX security symposium (USENIX Security 24). 2024: 1777-1794. [2] Du X, Wen M, Zhu J, et al. Generalization-enhanced code vulnerability detection via multi-task instruction fine-tuning[C]//Findings of the Association for Computational Linguistics: ACL 2024. 2024: 10507-10521. [3] Du X, Zheng G, Wang K, et al. Vul-rag: Enhancing llm-based vulnerability detection via knowledge-level rag[J]. ACM Transactions on Software Engineering and Methodology, 2024. [4] Lekssays A, Mouhcine H, Tran K, et al. {LLMxCPG}:{Context-Aware} vulnerability detection through code property {Graph-Guided} large language models[C]//34th USENIX Security Symposium (USENIX Security 25). 2025: 489-507. Analysis DL-based and fine-tuned PLM methods learn vulnerability-related features from labeled data, and therefore have relatively high Recall (0.765/0.824). However, these features are not fully consistent with the semantics of real vulnerabilities and are easily affected by suspicious but non-vulnerable semantics, so their Precision is relatively low (0.418/0.437). LLM-based and fine-tuned LLM baselines rely more on the model’s security knowledge, instruction understanding, and natural-language reasoning. When the vulnerabilities in the test samples are relatively new and lack sufficiently clear semantic evidence, they often make more conservative predictions; therefore, their Precision (0.433/0.432) and Recall (0.441/0.483) are both significantly lower than those of CodeDevia. Distributional Deviation Avg.Len Top1 Top2 Top3 Top4 Top5 Top6 Top7 Top8 Top9 Top10 Training 71.56 18.9% 44.9% 60% 70.1% 76.4% 81.4% 84.5% 87.2% 88.7% 90.1% Test 67.13 26.1% 54.1% 68.2% 75% 82.5% 89.6% 91.5% 93.2% 94.1% 94.3%
Fields of Study & Subjects
Software Engineering Research
Software Testing and Debugging Techniques
Web Application Security Vulnerabilities
Publication details
- Journal / source
- Zenodo (CERN European Organization for Nuclear Research)
- Publication date
- 27 March 2026
- Language
- EN