Artificial Intelligence - Partner in research, or the fifth wheel

Read full text Summarise with VIRAT
Authors: Lejo Buning, Jos van Hilligersberg, Peter C. Schuur, F.J. de Vijlder
Cited by: 0

Summary

No summary yet. VIRAT can read this paper and write a plain-language summary of its aim, methods, key findings and limitations.

Summarise with VIRAT Free to try with your Science Catalogs account.

Abstract

In social research qualitative analysis of interviews by verbatim coding is a quite common, yet a labour-intensive task. Artificial Intelligence, e.g. ChatGPT, may be helpful here.Previous research showed that qualitative researchers tend to self-monitor, especially when expertise is required. We also found that the application of Artificial Intelligence in qualitative analysis is unknown. The latter is reinforced by doubts on consistently trustworthy responses from the Artificial Intelligence systems studied. On trustworthiness we found that transparency and robustness are the two relevant components to address on this issue. As humans have, Artificial Intelligence may be found biased perhaps introduced by ‘previous experiences’; data used to train the Artificial Intelligence systems.This paper explores the performance in qualitative analysis of interviews of four generally available and interesting contemporary Artificial Intelligence systems: 1) Bard (Google), 2) Bing-AI (Microsoft), 3) ChatGPT (OpenAI), and 4) WriteSonic (WriteSonic). Therefore, we used one set of quotations to assess the summarizing performance of these four Artificial Intelligence systems.Over the four assess Generative AI systems we noticed wide variety on performance with respect to trustworthiness as well as on bias. Our research shows that ChatGPT may be a useful sparring partner in summarizing quotations emerging from interview analysis. Responses returned by ChatGPT showed a better performance on trustworthiness and a recognizable bias. In general, a supportive role could be seen for Generative AI as a co-pilot in coding and qualitative analysis. Whereas the response time in ample seconds, may be considered as a significant advantage. To conclude our study, we may argue that the variation in quality of ChatGPT responses, the use of ChatGPT can merely be considered as inspirational. The use of con-textually rich words in sentence structure also made us aware of a pitfall as the change of meaning of the sentence may occur.

Fields of Study & Subjects

Computational and Text Analysis Methods Qualitative Research Methods and Applications Ethics and Social Impacts of AI

Publication details

Journal / source
SocArXiv (OSF Preprints)
Publication date
25 March 2026
Language
EN
Document type
Preprint