Assessing AI-Based Code Generation using Code Comments

View at publisher
Authors: Thenmozhi Sekar, Ruby ElKharboutly
Cited by: 0
References: 13

Abstract

As Artificial Intelligence (AI) becomes an increasingly integral part of software development, modern AI tools such as ChatGPT, GitHub Copilot, Gemini, and DeepSeek are now widely used for code completion and generation. This research evaluates the effectiveness and limitations of these tools in using code documentation as input and generating the corresponding Python and Java code. Both the syntactic and semantic correctness of the generated code are assessed and compared with developer-written implementations, and analyzed using key software metrics such as Cyclomatic Complexity, Lines of Code (LOC), and runtime efficiency.Our results show that although AI-generated code is often comparable to developer-written code for simpler tasks, it varies in terms of structure, optimization, efficiency, and abstraction as the task complexity increases. The present study analyzed 200 code samples generated from documentation in both Java and Python. These results provide a realistic assessment of the strengths and weaknesses of existing AI-code generation tools as they reflect on the practical usefulness in real-world development workflows.

Fields of Study & Subjects

Software Engineering Research Software Engineering Techniques and Practices Artificial Intelligence in Law

Publication details

Journal / source
International Informatics and Software Engineering Conference (IISEC)
Pages
86–91
Publication date
5 February 2026
Language
EN