An exploratory study on LLM-generated code and comments in code repositories

The use of LLMs in software development has become increasingly widespread on tasks such as code generation and summarization. Reports from large technology companies showed that around 20% to 30% of their code are generated by LLMs. However, there remains skepticism about the practical usage of LLM-generated code and comments, such as concerns on more time for debugging the generated code and the unnaturalness of the generated comments. In this paper, we study the code and comments that were likely to be generated by LLMs and their characteristics, the differences between company- and community-maintained repositories, and how likely bugs are introduced by LLMs. We conduct extensive experiments on active company- and community-maintained repositories from 2021 to 2025 using various tools and techniques that detect code and comments generated by LLMs. We observed code detected as likely to be generated by LLMs decreased over time and was primarily in test cases, while that of comments, the proportion remains relatively similar over the years. We also observed code detected as likely to be generated by LLMs have code clones within their own repository, while comments detected as likely to be generated by LLMs show a low percentage of grammatically correct sentences. Furthermore, the company-maintained repositories show a higher percentage of code and comments detected as likely to be generated by LLMs, and only a small percentage of the human-labelled bugs found in the repositories are detected as introduced by LLMs.