Adversarial Training and Contrastive Loss: Addressing Biases in Non-Topical Classification and Text Regression Tasks

Topical shifts—differences between the topic distribution in training and test data—can substantially reduce the performance of non-topical text classification and regression models. In this work, we propose a modified sampling procedure for the contrastive loss to make it more robust to the topical shifts. We also investigate how to increase robustness to such shifts by combining Adversarial Domain Adaptation (ADA) with a modified contrastive learning objective. The method jointly minimizes label prediction loss, adversarial domain loss, and a contrastive loss designed to reduce dependence on topic-related confounders. We evaluate the approach on two tasks: sentiment prediction in Amazon Reviews and user education degree identification in the PASTEL dataset. Across both tasks, the proposed method improves robustness under topical shifts, reducing the mean absolute error in text regression and increasing the Quadratic Weighted Kappa (QWK) in text classification. These results demonstrate that jointly applying ADA and confounder-aware contrastive learning can mitigate topic sensitivity and yield more reliable text prediction models.