Generative AI in Corpus Linguistics: comparing ChatGPT-4 and manually developed NLP models for thematic concordance analysis

Corpus Linguistics (CL) has evolved through advances in Natural Language Processing (NLP) and generative AI tools like ChatGPT-4. While ChatGPT-4 excels in generating human-like language, its effectiveness in structured linguistic tasks such as thematic analysis remains uncertain. This study compares ChatGPT-4’s automated thematic classification of concordance lines with a specialised Long Short-Term Memory (LSTM) model using the same dataset. Results show ChatGPT-4 achieves reasonable precision (80–90%) in general thematic classification but struggles with context-sensitive themes, leading to notable misclassifications. These limitations are linked to ChatGPT-4’s reliance on probabilistic patterns rather than structured training. The study concludes that ChatGPT-4 is better suited as a complementary tool to structured NLP models, supporting refinement and interpretability, rather than serving as a standalone solution for complex linguistic analysis.