Feasible But Not Favoured: Comparing Conversational Interviewing in Chatbot vs Human-Administered Patient-Reported Outcomes for Stroke Survivors

Hospitals use patient-reported outcome measurements (PROMs) to monitor stroke survivors post-discharge.Unfortunately, PROM items are prone to misinterpretation.Human-administered PROMs mitigate comprehension issues by providing clarifications through conversational interviewing (CI).Using the four-step model of survey response as a framework, we hypothesise that chatbots can utilise CI to support users at every stage of the question-answer process.We conducted a controlled experiment with 18 stroke survivors comparing two CI clarification techniques, 1) additional examples and 2) reflective prompts against 3) no clarification (control), administered by both a chatbot and a human interviewer.Although participants preferred the human interviewer, their responses remained consistent with those given to the chatbot.Participants' preferred clarification varied by question type: favouring examples for questions regarding physical activity, and no clarification for mental health-related items.Our findings highlight the feasibility of chatbots utilising CI to administer PROMs while identifying four design challenges -personalising clarification formats, managing user expectations, encouraging disclosure, and ensuring safety -that must be addressed to create an effective, ethical, and trustworthy PROM-administering chatbot.