AutoLabel-ABSA: A Comparative Study of Transformer Embeddings and Clustering Algorithms for Automatic Text Labeling and Aspect-based Categorization

Automated text labeling is a promising approach to reduce annotation costs in natural language processing, particularly for aspect-based sentiment analysis and topic classification.This paper presents AutoLabel-ABSA, a systematic framework that combines state-of-the-art transformer embeddings BERT, RoBERTa, and BigBird with multiple clustering algorithms HDBSCAN, K-Means, Agglomerative, and Spectral Clustering to generate high-quality pseudo-labels from unlabeled text.We evaluate the pipeline on three benchmark datasets: IMDB, AG News, and 20 Newsgroupss.Results show that RoBERTa paired with HDBSCAN yields the most coherent clusters, achieving strong downstream classification accuracy (up to 94.5% on IMDB).Notably, BigBird offers minimal gains over RoBERTa despite higher computational cost, suggesting that long-context modeling is unnecessary for standard-length texts.Our findings provide practical guidance for unsupervised labeling in low-resource settings.