Mix-And-Match: Targeted Data Reconstruction Attacks Against LLM-Based Text Classification Models

Natural Language Processing (NLP) models have achieved widespread adoption in diverse realworld applications, notably in text classification tasks. Despite their utility, these models remain susceptible to privacy threats, particularly data reconstruction attacks that seek to recover sensitive training data from deployed models. While the majority of prior research has concentrated on the vulnerabilities of Large Language Models (LLMs) to such attacks, classification models have traditionally been regarded as comparatively secure. In this study, we introduce a novel targeted data reconstruction attack, termed the Mix And Match attack, which exploits the architectural dependence of modern classification models on LLM backbones. The proposed attack operates by leveraging the base LLM of the target classification model to generate candidate token sequences, subsequently refining these candidates through the model's classification head to maximize reconstruction fidelity. We conduct comprehensive empirical evaluations of the Mix And Match attack, employing both synthetic (random) and naturally occurring (organic) canary tokens to assess its efficacy across multiple classification benchmarks. Our results demonstrate that classification models, contrary to prevailing assumptions, are vulnerable to significant privacy leakage via data reconstruction, especially when their underlying LLMs are accessible or partially exposed. This work underscores the critical need to reassess privacy risk models for NLP classification systems and provides actionable insights into the mechanisms and extent of potential data leakage. We further discuss mitigation strategies and the broader implications for secure deployment of NLP models in sensitive domains.