Is genAI a good editor of academic writing?

This article is about the performance of generative AI (genAI) as an editor. It reports on a study of eight genAI platforms’ outputs when genAI was asked to edit two student and two professional academic writing samples. In the article, we analyze whether and to what extent different versions of Anthropic’s Claude, Google’s Gemini, Microsoft’s Copilot, and OpenAI’s ChatGPT found twenty types of sentence-level errors as well as how their performance compared to human editing. Results show genAI did not catch as many errors as the human baseline as well as over- and under-identified certain kinds of sentence-level errors. Variables among writing samples (e.g., stage of writing, presence of rubric) did not affect how many errors were identified but did influence which types of errors were identified. Based on this study, we argue people cannot yet gain efficiency by using most genAI platforms as an editor if they want to achieve the same level of accuracy as humans. Relying on genAI to edit texts may also lead to over-correction of structural issues and under-correction of contextually-dependent errors.