Evaluating LLM-Based Bias Detection on Wikipedia: A Prompt Strategy Analysis Using NPOV Guidelines

Wikipedia’s Neutral Point of View (NPOV) policy is central to maintaining the integrity of one of the world’s most widely accessed information sources. As AI-powered moderation tools become increasingly prevalent in adaptive web environments, understanding their reliability and limitations is critical. This paper investigates the effectiveness of Large Language Models (LLMs) in detecting neutrality violations in Wikipedia text using the Wikimedia Neutrality Corpus (WNC). We evaluate three Claude models—claude-3-haiku, claude-haiku-4-5, and claude-sonnet-4-5—across four prompt strategies ranging from direct instruction to policy-guided few-shot prompting. Performance is further analyzed across topical domains, macro-topic categories, bias types (framing, demographic, and epistemological), and discourse communities. Our results show that claude-sonnet-4-5 with NPOV guidelines and detailed explanations achieves the highest accuracy of 69.0%, yet performance varies substantially across domains—medical discourse showing the highest disagreement rate at 64.3%. These findings reveal important risks and limitations of deploying LLMs for automated content moderation on the adaptive web, and offer practical guidance for prompt design in policy-grounded classification tasks.