A qualitative study with patients, care-partners, clinicians, and bioethicists to identify ethical considerations of artificial intelligence tools in palliative care

BACKGROUND: The use of artificial intelligence (AI) in medicine has surged. Given the sensitive nature of palliative care, it is crucial to apply AI tools in a patient-centered and ethical manner. AIM: We sought to understand the ethical implications of implementing AI-based tools in palliative care from various stakeholders. DESIGN: Using a framework of ethics of care, we conducted thematic analysis to identify emerging themes. SETTING/PARTICIPANTS: We recruited and interviewed 22 participants: six patients, four care-partners, five clinicians, three bioethicists, and four clinician-bioethicists, recruited through a single hospital clinic for patients and care-partners and purposeful snowball sampling for clinicians and bioethicists based on their knowledge and interest in AI-based tools. The mean age of participants was 48 years old; 64% were male. Clinician participants had practiced for a mean of 11 years, and ethicists for 15 years. We aimed to recruit clinicians and bioethicists from multiple states and specialties. RESULTS: From the interviews we extracted five main themes: (1) Primacy of the doctor-patient relationship over AI performance; (2) Humans have intuition and nuance that AI lacks; (3) Agreement about the importance of oversight of AI tools; (4) New AI technologies should include a process for patient education; and (5) AI increases efficiency, scalability, and a more unified approach to serious illness. CONCLUSIONS: When building and implementing AI-based tools, we recommend: establishing oversight committees; reflecting on the unique contributions of humans to care; proactively educating patients and contextualizing the tools; and ensuring data use is restricted to clinical care.