LLM-based voice chatbot surveys as an alternative to post-experience questionnaires: probe-controlled, ultra-short field interviews
Abstract
ABSTRACT: Chatbot-based surveys offer low-burden, in-situ data collection, yet unconstrained LLMs often drift from research aims. We conducted 359 ultra-short, post-experience voice interviews in a public venue to compare a framework-guided LLM, an unconstrained LLM, and fixed questions. The guided approach produced significantly longer responses than fixed questions and yielded the richest diversity of process-specific accounts. These findings show that probe control is essential for eliciting actionable, experience-grounded feedback in real-world, time-limited settings.
Fields of Study & Subjects
AI in Service Interactions
Digital Mental Health Interventions
Artificial Intelligence in Healthcare and Education
Publication details
- Journal / source
- Proceedings of the Design Society
- Publisher
- Cambridge University Press (CUP)
- Volume
- 6
- Pages
- 2253–2262
- Publication date
- 2 July 2026
- Language
- EN