Uncertainty-Aware Contrastive Sentence Embedding With Local Context Representation for Text Classification

In real-world applications of natural language processing, it is essential to effectively adapt a pre-trained model to a downstream task.While text classification is undertaken as a downstream task, it is crucial to produce meaningful sentence embedding that is adaptive to the task.In this paper, we explore how to effectively adapt a pre-trained model for extracting meaningful context representations from sentences, and propose an uncertainty-aware contrastive sentence embedding approach that involves addressing language ambiguity and inter-class separability for a text classification task.Specifically, we design an end-to-end strategy for driving the process of learning to transform a word embedding matrix into a contextualized sentence vector and to quantify the representation uncertainty of the sentence, while the word embedding matrix is produced by a pre-trained model without fine-tuning, and a label-wise contrastive learning strategy is designed to enhance intra-class compactness and inter-class separability.The results on public data sets show that a considerable improvement of text classification accuracy is achieved by adopting the proposed approach in comparison with using those state-of-the-art methods 1 .