Developing a Generative AI-Based English Speaking Chatbot with Level-Specific Feedback: Effects on Speaking Performance and Anxiety

This study developed and examined the effectiveness of a generative AI–based English speaking learning support chatbot designed to enhance adult learners’ speaking proficiency and reduce speaking anxiety through level-specific feedback. The chatbot employed a prompt structure combining an evaluation mode and a conversation mode, enabling it to automatically assess learners’ speaking proficiency and provide feedback tailored to their proficiency levels. The experiment involved 19 Korean undergraduate and graduate students who used the chatbot over a two-week period, employing a single-group pretest–posttest design. Based on pretest speaking scores, participants were categorized into novice, intermediate, and advanced groups. The results showed that all three proficiency groups demonstrated statistically significant improvements in speaking proficiency, while a statistically significant reduction in speaking anxiety was observed only in the novice group. Discourse analysis indicated that the chatbot consistently delivered level-specific feedback, although learners’ immediate response rates to the feedback were relatively low. These findings suggest that chatbot-based repetitive speaking practice combined with automated level-specific feedback can effectively support adult learners’ speaking development and highlight the potential of future generative AI–based chatbots in educational settings.