AI-Detection Bias and False Positives: Comparing 2016 Human, 2026 AI, and 2007 Student Essays Across Common Detectors

AbstractThe integration of generative language models into academic and professional settings has driven adoption of AI-detection software. Universities use these tools to assess authorship, but their reliability remains contested. This study evaluates several common detectors by comparing their assessments of three texts: a 7,062-word polished human essay (2016), a 1,100-word AI-generated essay (2026, mixed formal–informal), and a 900-word seventh-grade human essay (2007). Results show extreme variability across detectors, substantial false positives on human writing, and high inter-tool disagreement. Notably, the 2016 human essay scored higher for AI than the AI-generated essay. These findings raise serious concerns about institutional reliance on AI detectors for academic judgment.Revised ConclusionAppendix C shows that structured writing, whether semi-formal or even informal with proper grammar and structure, produces misclassification rates that approach parity. A little less than 50% of clearly structured human-written text is identified as AI, whereas almost the same percentage of AI-generated material is identified as human. In effect, once structure is established, detection accuracy approaches random performance. The probability of misclassification becomes statistically equivalent to a coin flip.