Privacy at the Core: Toward Automated Detection of Privacy-Sensitive Content in an LLM-Based Care Documentation Support System

Large language models (LLMs) introduce new opportunities in residential care, including the potential to assist with care documentation. However, if introduced unreflected, such technologies present challenges and potential harms to privacy and personal integrity. In this paper, we present a framework for automated filtering of privacy-sensitive content from LLM-supported care documentation. Our framework is based on Nissenbaum's theory of privacy as contextual integrity. As an initial step, we present the generation of a synthetic dataset derived from privacy-sensitive interactions between care workers and care recipients in the real world. We analyze the conversations by privacy categories and show that both care recipients and care workers are affected. Our contributions include a methodology for generating privacy-preserving synthetic datasets and insights into the content requirements of a dataset for fine-tuning an LLM to detect privacy-sensitive segments. In addition, we show that value-sensitive design can result in innovative approaches to creating technology that is safe, meaningful, and protective of important human values.