Enhancing Data Privacy in Large Language Models Through Private Association Editing

Large language models (LLMs) require a significant redesign in solutions to preserve privacy in data-intensive applications due to their text-generation capabilities. Indeed, LLMs tend to memorize and emit private information when maliciously prompted. In this paper, we introduce Private Association Editing (PAE) as a novel defense approach for private data leakage. PAE is designed to effectively remove Personally Identifiable Information (PII) without retraining the model. We experimented on three open-weight, open-data models—GPT-Neo 1.3B, GPT-Neo 2.7B, and GPT-J—by applying Training Data Extraction (TDE) attacks to retrieve hundreds of PII, including email addresses, phone numbers, and Twitter handles. Since these models were trained on Pile, an openly available pre-training dataset, it is possible to verify the true extent of the data leakage. While all three models tend to leak PII under TDE attacks, experimental results demonstrate the effectiveness of PAE with respect to alternative baseline methods in defending against those attacks. In fact, unlike other techniques that tend to degrade model performance, our experiments show that PAE consistently reduces the number of leakages without affecting the model’s utility. We believe PAE will serve as a practical tool for removing memorized PII from deployed LLMs without retraining.