Impact of Synthetic Review Filtering on the Performance of Sentiment Analysis: A Category-wise Analysis Using BERT+CRF

This study examines the impact of synthetic reviews, which have proliferated with the rapid adoption of generative AI, on the performance of sentiment analysis models in online review environments. Although AI-generated reviews often exhibit fluent and coherent language, they may fail to reflect genuine user experiences, potentially distorting training data distributions and degrading model performance. To investigate this issue, we conduct a comparative experiment using an AI Hub(https://aihub.or.kr/) aspect-based sentiment analysis dataset composed of real user reviews. Synthetic reviews generated under fixed conditions are selectively included or excluded from the training data to evaluate the effect of synthetic review filtering. Experiments are performed across five review categories—home appliances, IT devices, daily goods, cosmetics, and fashion—using a BERT+CRF-based model as the primary architecture, with an LSTM-based model as a baseline. The results show that removing synthetic reviews consistently improved sentiment analysis performance, with a notable increase in Macro-F1 scores. Moreover, the magnitude of improvement varies by review category, with larger gains observed in information-oriented domains such as home appliances and IT devices. These findings suggest that domain characteristics play a crucial role in the effectiveness of synthetic review filtering. Overall, this study provides empirical evidence that data refinement strategies are essential for enhancing the reliability and robustness of sentiment analysis systems in the era of generative AI.