Explaining in context: Perceived informativeness of Explainable Artificial Intelligence (XAI) in Arabic hate speech detection

Hate speech is pervasive across most social media platforms, profoundly impacting users’ lives and wellbeing. To address this critical issue, researchers are exploring explainable AI (XAI), which helps users understand AI decision-making and outputs. To ensure its effectiveness, XAI must align with users’ expectations, often assessed using user-centered approaches from human-computer interaction (HCI). However, the integration of XAI in hate speech detection, particularly in Arabic contexts, remains underexplored. This study proposed and evaluated a high-fidelity prototype of an Arabic hate speech detection system with explainable features. The focus was on assessing the informativeness of different XAI representation methods and how Arabic-speaking users perceived them. Both subjective and objective measures were used, including the Social Media Activity Questionnaire (SMAQ), Social Networking Usage Questionnaire (SNUE), a Perceived Informativeness (PI) questionnaire, and eye-tracking to assess visual attention. Results showed that users perceived both textual and visual explanations—specifically visual saliency—as more informative than other XAI methods. Social media usage duration had no significant effect on perceived informativeness. While the combination of textual and visual explanations (presented as a pie chart) resulted in longer fixation durations, this did not necessarily translate into higher perceived informativeness. The extended fixation was likely due to the additional visual element rather than increased clarity. These findings highlight the importance of designing visually clear and focused XAI representation methods. Elements like text highlighting through saliency can improve user perception of informativeness more effectively than simply adding visual complexity. • Participants preferred the combination of textual explanations and highlighted text over other XAI representation methods. • Social media usage duration did not significantly influence perceived informativeness score across different user interfaces. • Fixation duration does not necessarily influence perceived informativeness. Visual elements can influence attention allocation, they do not always enhance users’ perception of informativeness.