What do we gain(and lose) from responsive questions? Comparing open-ended survey questions with adaptive probes from an AI interviewer

This study examines how interface- and adaptiveness changes the quality of data generated from semi-structured interviews. In a randomized controlled trial with approximately 800 to 1200 adults, respondents are asked to reflect on the deservingness of strangers in need using five core items based on the CARIN framework. Participants are assigned to one of four treatment arms, or interview modes: a standard survey, a standardized chat interface, an AI interviewer with one adaptive follow-up, or an AI interviewer with multiple adaptive follow-ups. This design isolates the effects of interface and adaptiveness of the interviewer. Comparing arm 1 and 2, we can estimate the effect of interface, and comparing and 2, 3 and 4 we can estimate the effect of adaptiveness of the interviewer. We create several outcome measures to capture the various factors that impact the overall assessment of the quality of the interview data. We present pre-specified hypotheses we intend to examine and test statistically. We refer to the included pre-analysis plan for details.