Automated code comments generation using large language models: Empirical evaluation of T5 and bart
Abstract
Source code documentation is an essential part of the software development process because it helps make programs easier to understand, maintain, and use over time. However, many developers either skip writing comments or produce ones that do not meet professional standards. With the rapid progress of Artificial Intelligence, especially in Large Language Models (LLMs), new methods have emerged to automatically generate code comments to improve documentation quality, consistency, and efficiency. This empirical analysis explores how effective two popular open-source LLMs, T5, created by Google AI, and BART, developed by Facebook AI, are at generating single-intent code comments for Python and Java programs. The research was carried out in two experimental phases using carefully selected datasets collected from public GitHub repositories. In the first phase, the models were tested using lexical similarity measures such as BLEU, ROUGE, METEOR, and Smoothed BLEU to assess fluency and accuracy. The second phase extended the evaluation to include the Summary alIgnment to coDe sEmantics (SIDE) metric, enabling a deeper analysis of semantic alignment and contextual understanding between the source code and the generated comments. Experimental outcomes revealed that BART consistently outperformed T5 across most evaluation metrics, producing comments that were more fluent, accurate, and semantically aligned with real-world software code. T5, however, demonstrated greater computational efficiency, making it suitable for resource-limited environments. Moreover, these findings indicate that open-source, lightweight, and resource-efficient models such as BART and T5 provide practical, transparent, and scalable solutions for automating software documentation. The exploration also establishes a foundation for future research on multi-intent comment generation and advanced semantic evaluation frameworks to further bridge the gap between human and machine understanding of code.
Fields of Study & Subjects
Software Engineering Research
Topic Modeling
Natural Language Processing Techniques
Publication details
- Journal / source
- Melbourne Institute of Technology
- Publication date
- 12 February 2026
- Language
- EN