Assessing AI-Based Code Generation using Code Comments
Abstract
As Artificial Intelligence (AI) becomes an increasingly integral part of software development, modern AI tools such as ChatGPT, GitHub Copilot, Gemini, and DeepSeek are now widely used for code completion and generation. This research evaluates the effectiveness and limitations of these tools in using code documentation as input and generating the corresponding Python and Java code. Both the syntactic and semantic correctness of the generated code are assessed and compared with developer-written implementations, and analyzed using key software metrics such as Cyclomatic Complexity, Lines of Code (LOC), and runtime efficiency.Our results show that although AI-generated code is often comparable to developer-written code for simpler tasks, it varies in terms of structure, optimization, efficiency, and abstraction as the task complexity increases. The present study analyzed 200 code samples generated from documentation in both Java and Python. These results provide a realistic assessment of the strengths and weaknesses of existing AI-code generation tools as they reflect on the practical usefulness in real-world development workflows.
Fields of Study & Subjects
Software Engineering Research
Software Engineering Techniques and Practices
Artificial Intelligence in Law
Publication details
- Journal / source
- International Informatics and Software Engineering Conference (IISEC)
- Pages
- 86–91
- Publication date
- 5 February 2026
- Language
- EN