A Clinician-Led Governance Framework for Evaluating Behavioral-Health AI Communication Safety

Abstract Conversational artificial intelligence systems are increasingly used in behavioral-health contexts, where users frequently express emotional distress, uncertainty, and culturally nuanced needs. Although general AI governance frameworks provide high-level principles for responsible development, they do not offer domain-specific mechanisms for evaluating the safety and appropriateness of behavioral-health AI communication. This study introduces a clinician-led governance framework designed to evaluate communication-level safety in behavioral-health AI systems. The framework includes ten interdependent governance domains constructed through clinical analysis, socio-technical review, and iterative conceptual refinement. Inter-rater reliability (IRR) procedures were conducted using trained behavioral-health reviewers scoring a standardized set of AI outputs across all domains. Agreement metrics demonstrated substantial to near-perfect consistency. Across domains, Cohen’s κ values ranged from moderate (0.64) to near-perfect agreement (0.89). This work addresses an urgent oversight gap in AI governance by providing a structured, clinically informed method for evaluating the safety of behavioral-health AI communication.