Improving Automated Feedback Systems for Tutor Training in Low-Resource Scenarios Through Data Augmentation

Tutoring is an effective instructional method for enhancing student learning, yet its success relies on the skill and experience of the tutors. This reliance presents challenges for the widespread implementation of tutoring, particularly in training novice tutors. To support tutor training programs, real-time automated feedback systems are essential for efficiently training large numbers of tutors. A recent study employed Generative Pre-Trained Transformers (GPT) for sequence labeling to identify desirable and undesirable praise components in a tutor training dataset, providing explanatory feedback. However, this approach requires a significant amount of labeled data for fine-tuning, which is both labor-intensive and dependent on expert input. To address the challenges associated with extensive data labeling, the current study explores the use of prompting more advanced GPT models such as GPT-4o to generate synthetic datasets for augmenting labeled response data, followed by fine-tuning a GPT-3.5 model. Our results show that fine-tuning GPT-3.5 on a GPT-4o-generated augmented dataset of 520 instances yields statistically meaningful improvements in the identification of praise components. Specifically, the$F_{2}$score increases by 20.5% for effort-based praise and by 17.3% for outcome-based praise relative to the same model fine-tuned without data augmentation. Moreover, the proposed data augmentation approach exhibits effective generalizability to the identification of additional praise categories, specifically person-based praise. When fine-tuned on an augmented dataset of 520 instances, the model achieves a 16.5% improvement in the$F_{2}$score for person-based praise, accompanied by increases of 19.9% in M-IoU and 21.6% in IoU, relative to the same model fine-tuned without data augmentation. These findings suggest that for data-intensive tasks, synthetic data generated through GPT model prompting can substantially enhance fine-tuned model performance in low-resource scenarios. This approach offers a scalable and cost-effective solution to alleviate the reliance on extensive expert-labeled data, enabling broader adoption of automated tutor training systems and facilitating the rapid development of skilled tutors at scale.