The Detectability of Partially AI-Rewritten Academic Documents from Stylometric Features

Working paper and open per-document data. Real documents are often partly AI-processed, with a few sections AI-rewritten and the rest left as human-written. This study measures how a stylometric human-versus-AI classifier responds as the AI-rewritten share of a document increases from 0 to 100 percent. From 25,561 content-aligned pairs of human-written academic texts and their AI rewrites, we splice documents in which a controlled fraction of the words, in increments of 10 percent, has been rewritten by one of eight AI model configurations, and score all 382,993 resulting documents with a classifier trained only on fully human and fully AI-rewritten texts, under grouped cross-validation throughout. The classifier reads the AI share as a proportion on a continuum, rather than detecting its presence. The mean score increases almost linearly from 0.17 to 0.83, and the share of documents flagged at the default threshold crosses 50 percent almost exactly at half AI content. Scattering the AI sentences through the document instead of concentrating them in one block leaves the response unchanged. Small AI shares are close to invisible. A document with 10 percent of its words AI-rewritten is separated from fully human documents with an AUC of 0.57, barely above chance, and under a strict one percent false positive budget only 15 percent of half-rewritten documents are flagged. Every feature mixes near linearly with the AI share except sentence-length variation, which is above the value linear mixing predicts, since the style change at the splice point itself adds variation, and 36 percent of half-rewritten documents show more sentence-length variation than the original human text. Distinguishing half-rewritten documents from pure ones as a class reaches an AUC of only 0.59 for a linear model and 0.76 for gradient boosting, so partial rewriting is detectable mainly as a lower score, not as a recognizable discrete category. Version 1.1 adds the interleaved rewriting experiment, in which every second sentence of the human text is replaced by the position-matched sentence of its AI rewrite. Interleaved documents at approximately half AI content score exactly on the block-splice response curve, so the whole-document score responds to the amount of AI text and not to its arrangement. The full paper is published at https://textpulse.ai/research/ai-partial-rewriting-detection. This record holds the paper PDF, the per-document out-of-fold classifier scores for all 382,993 pure, spliced, and interleaved documents (opaque identifiers only, no text), the per-share metrics, and all analysis and figure code.