Structural Prompt Variants and Their Impact on LLM-Based Vulnerability Detection Accuracy

Read full text Summarise with VIRAT
Authors: Javier Martínez, Alejandro Ruiz, Mariana Torres
Cited by: 0
References: 15

Summary

No summary yet. VIRAT can read this paper and write a plain-language summary of its aim, methods, key findings and limitations.

Summarise with VIRAT Free to try with your Science Catalogs account.

Abstract

The structure of prompts used in vulnerability detection tasks significantly affects the reasoning process of large language models. This study systematically evaluates how formatting,decomposition, and semantic density influence the accuracy of detecting five major vulnerability categories (CWE-20, CWE-79, CWE-89, CWE-120, CWE-798). We construct a benchmark of 12,000 prompt–code pairs and test three LLMs: GPT-4, Claude-3, and Llama-3-70B. Results show that structured prompts with explicit reasoning stages improve true-positive rates by 24–41%, while overly verbose prompts increase hallucination-induced false alarms by 15%. Prompt–model mismatch analysis reveals that models differ in tolerance to prompt ambiguity. These findings highlight the necessity of prompt normalization for trustworthy security auditing using LLMs.

Fields of Study & Subjects

Web Application Security Vulnerabilities Software Engineering Research Adversarial Robustness in Machine Learning

Publication details

Journal / source
American Journal of Artificial Intelligence and Neural Networks
Volume
7
Issue
1
Pages
124–132
Publication date
28 February 2026
Language
EN