From GPT-4 to GPT-5 The Impact of LLM Generation on Spanish Chat Interactions

This repository contains the data, configurations, and statistical analysis outputs for the empirical study on how different LLM generations impact human-AI interaction in foreign language learning # 01_Technical_SetupThis folder contains the technical configurations for the Conversational Agents exported as JSON files from the Flowise platform.- System Prompts: Full metaprompt instructions for the personas Carla, Samir, and Ramirez.- Model Parameters: API settings including specific version bindings for the research.- Model Versions: Configurations for OpenAI gpt-4-0613, gpt-4-turbo-2024-04-09, and gpt-5-2025-08-07. # 02a_Raw_Chatlogs.xlsxThis file contains the anonymized chat corpus used for linguistic analysis.* Scope: 159 conversations with a total of 2,007 messages.* Data Privacy: All participant names and identifying details have been pseudonymized. # 02b_User_Ratings.xlsxThis file contains the subjective evaluations provided by the participants.* Sample Size: 144 complete evaluations from 48 participants.* Scale: Scores range from 1 (very good) to 5 (very bad). # 03_Analysis_OutputsThis folder contains the results of the Mixed Model Analysis (MLA) exported from SPSS.* Formal Metrics (RQ1): Frequency analysis for emojis, exclamation marks, and question marks.* Behavioral Engagement (RQ2): Analysis of conversation duration, response latency, and character counts.* Perceived Quality (RQ3): Statistical comparisons of user scores across model generations.Files: - conv_duration.pdf = coun_characters_message_LLM.pdf - count_characters_message_user.pdf - count_emoji_LLM.pdf - count_emoji_user.pdf - count_exclamation_marks_LLM.pdf - count_exclamation_marks_user.pdf - count_question_marks_LLM.pdf - count_question_marks_user.pdf - rating_result.pdf ## Study Methodology * Framework: Task-Based Language Teaching (TBLT) using an "opinion-gap" negotiation scenario.* Design: Within-subject design where each participant interacted with all three LLM versions.* Population: Students from German secondary schools (8th & 12th grade) and university-level learners (B1/B2). ## License and CitationThese data are provided for replication and transparency purposes in Open Science.* Citation: Please cite the original paper: "Fahnroth, Felix (2027) "From GPT-4 to GPT-5: The Impact of LLM Generation on Spanish Chat Interactions."