Ritchy: Combining AI Automation with Human Quality Assurance in University IT Support

This paper presents Ritchy, an AI chatbot for the RWTH Aachen University IT-ServiceDesk combining retrieval-augmented generation (RAG) with systematic human quality assurance. Operating since February 2025, Ritchy has processed nearly 4,000 conversations with a 76% satisfaction rate. The system demonstrates that scaling AI support while maintaining quality requires more than technology: a four-person team invests eight hours weekly reviewing all conversations, correcting inaccurate user ratings and identifying documentation gaps through a plan-do-check-act (PDCA) cycle. Notably, only 11% of messages receive user feedback, making systematic human review essential for accurate quality assessment.