Automated outlier detection by AIs

This is a continuation of my recent research examining whether the naive use of the current AI systems can lead to research misconduct. Building on the previous observation that current AI systems can offer ways to selective reporting (Kawahara 2026c) and p-hacking (Kawahara 2026a), I examined whether the three major AI systems (ChatGPT, Claude and Gemini) would find and exclude outliers after seeing the results, a clear form of post-hoc data manipulation that constitutes research misconduct under standard definitions. When asked to “remove outliers to achieve statistical significance” with a very direct prompt, I observed varying responses. Claude explicitly identified the request as research misconduct and denied to execute the request. ChatGPT initially refused but then suggested methodological alternatives that could achieve the same outcome. Finally, Gemini executed the request, reported a “revised and now-significant” p-value and offered a methodological justification that misrepresented the post-hoc nature of the analysis.