A Structured Comparison of the Coalition for Health AI Responsible AI Guide and South Korea’s Trustworthy AI Guideline for Health Care AI Assurance: Comparative Framework Analysis

Background: Trustworthy artificial intelligence (AI) in health care requires assurance frameworks that translate ethical principles into measurable governance and evaluation practices. While a growing number of AI assurance frameworks have been proposed, they differ substantially in governance structure, institutional embedding, and implementation mechanisms, reflecting differences in intended purpose and use. To date, few studies have applied standardized, rubric-based evaluation criteria to systematically compare how assurance instruments with different institutional origins operationalize ethical principles across the AI lifecycle. Objective: This study aimed to develop and apply a structured, rubric-based evaluation instrument to compare 2 health care AI assurance instruments, including the Coalition for Health AI (CHAI) responsible AI guide, a voluntary consortium-based instrument, and South Korea's Trustworthy AI guideline, a government-issued instrument. Methods: A 7-dimension evaluation rubric was developed based on a synthesis of established international AI assurance and governance instruments. The rubric covered core principles, AI lifecycle coverage, governance context, stakeholder breadth, operational maturity, instrument design and tools, and public accessibility. Seven independent evaluators with expertise in health care AI governance assessed each instrument using a 5-point ordinal rating scale (1=absent-5=comprehensive). Each evaluator independently scored the materials using a standardized rubric. Discrepancies were resolved through structured consensus discussions, with reference to rubric definitions and source documents. Final scores were determined based on documented evidence, requiring full consensus rather than averaging. Interrater reliability was assessed using Fleiss kappa. Results: Both instruments demonstrated strong alignment in core principles (CHAI: 4; Trustworthy AI Guideline: 5) and stakeholder breadth (both: 4). The government-issued Trustworthy AI Guideline exhibited broader AI lifecycle coverage (5 vs 4), a more formalized governance context (5 vs 3), and higher operational maturity (4 vs 2), reflecting stepwise oversight and formal embedded oversight mechanisms supported by legislation. In contrast, the voluntary CHAI instrument demonstrated greater emphasis on instrument design and implementation tools (4 vs 3) and higher public accessibility (5 vs 3), driven by open-access resources such as assurance standards guides and applied model cards. Interrater agreement of independent ratings was moderate to substantial (Fleiss kappa=0.47-0.64; P