Evaluation of GRU with Attention and DistilBERT in Text Classification Tasks

In this paper compares two deep learning models for multiclass text classification in the cyberbullying domain. The corpus is written in English and includes six classes representing different types of harassment. The trained models are a GRU network with an attention mechanism and the DistilBERT transformer. The aim applies to a 5-fold cross-validation scheme. And uses accuracy, precision, recall, and F1 as evaluation metrics. The GRU model achieves significantly higher performance (accuracy = 0.83, F1 = 0.45) than DistilBERT (accuracy = 0.018, F1 = 0.004). The results obtained indicate that the GRU architecture with word embedding performs more effectively on this task. This is due to the dataset’s linguistic characteristics, as well as the limited fine-tuning capacity of the lightweight transformer. The findings emphasize the need to select model architectures that align with the corpus properties and the classification task’s complexity. Future work will involve fine-tuning strategies and evaluating additional transformer-based models. Smart citations: https://scite.ai/reports/10.61467/2007.1558.2026.v17i2.1268Dimensions.Open Alex.