EdgeWisePersona: A Dataset for On-Device User Profiling from Natural Language Interactions

This paper introduces EdgeWisePersona, a novel dataset and evaluation benchmark for assessing and improving small, edge-deployable language models for user profiling from multi-session smart home interactions. The dataset features structured user profiles defined by contextual routines, paired with natural language dialogues. The primary task is profile reconstruction: inferring user routines and preferences solely from interaction history. To assess how well current models can perform this task under realistic conditions, we benchmarked several state-of-the-art compact language models and compared their performance against large foundation models. Our results show that while small models demonstrate some capability in reconstructing profiles, they still fall significantly short of large models in accurately capturing user behavior. This performance gap poses a major challenge - particularly because on-device processing offers critical advantages, such as preserving user privacy, minimizing latency, and enabling personalized experiences without reliance on the cloud. By providing a realistic, structured testbed for developing and evaluating behavioral modeling under these constraints, our dataset represents a key step toward enabling intelligent, privacy-respecting AI systems that learn and adapt directly on user-owned devices.