TriCon-Fair: Triplet Contrastive Learning for Mitigating Social Bias in Pre-Trained Language Models

Pre-trained language models (PLMs) can amplify social biases and produce harmful, unfair outputs. Many debiasing methods either treat biased/unbiased samples independently or optimize standard contrastive objectives (e.g., InfoNCE) whose shared normalization couples gradients across samples. In this setting, repelling biased negatives can unintentionally disrupt positive alignment, creating a bias–utility trade-off. We propose TriCon-Fair to break this coupling. TriCon-Fair builds counterfactual triplets by pairing each anchor with an unbiased positive and an explicitly biased hard negative generated by prompting a frozen LLM, then optimizes attraction and repulsion as two independent (decoupled) terms. An auxiliary language-modeling objective is trained jointly to mitigate forgetting and preserve general capability. On StereoSet, TriCon-Fair reduces bias across both masked (encoder-only) and causal (decoder-only) PLMs, including BERT, GPT-2, and LLaMA-2, while maintaining utility. For example, it improves GPT-2 ICAT by up to 8.0 points over the original model with GLUE performance essentially unchanged. These results position TriCon-Fair as a compact and architecture-agnostic debiasing approach for fairness-critical NLP applications.