From Physical Activity to Online Engagement: On-Device Large Language Models that Predict Day-to-Day Mental Wellbeing (Preprint)

BACKGROUND Mental wellbeing is shaped by complex interactions among behavioural, social, and environmental factors. Traditional mental health assessment methods are episodic and resource intensive, limiting their ability to capture day-to-day fluctuations. The widespread use of smartphones and social media enables continuous, passive monitoring of both physical-world and virtual-world behaviours, offering new opportunities for scalable and privacy-aware mental wellbeing assessment. OBJECTIVE This study aimed to develop and evaluate a multimodal framework for predicting daily mental wellbeing by integrating smartphone sensor data and social media engagement logs using lightweight, on-device large language models (LLMs), and to assess the trade-offs between prediction accuracy, model size, and computational efficiency. METHODS Ten UK-based participants were recruited for a one-month longitudinal study. A custom Android application passively collected multimodal data, including physical activity, location, audio context, phone and app usage, and social media logs from Instagram, YouTube, and TikTok. Daily mental wellbeing was measured using ecological momentary assessments based on the 14-item Warwick–Edinburgh Mental Well-Being Scale (WEMWBS). Four open-source LLMs (Llama3.2-1B/3B and Gemma3-1B/4B) were evaluated using zero-shot prompting, few-shot prompting, retrieval-augmented generation (RAG), and parameter-efficient instruction fine-tuning with 4-bit quantisation. Model performance was assessed using mean absolute error (MAE), token-level output accuracy, and GPU memory usage. RESULTS Instruction fine-tuned models consistently outperformed zero-shot, few-shot, and RAG-based approaches across all data modalities. Fine-tuned LLMs achieved MAEs of approximately 2.6-3.3 on the 14-70 WEMWBS scale with token accuracy exceeding 95%. Combining physical-world and virtual-world behavioural data resulted in an approximately 10% relative reduction in MAE compared to single-modality inputs. Notably, the fine-tuned 1B-parameter Llama3.2 model matched or exceeded the performance of larger models while requiring substantially lower computational resources. CONCLUSIONS Multimodal integration of physical and virtual behavioural data substantially improves daily mental wellbeing prediction. Lightweight, fine-tuned LLMs can deliver high predictive accuracy while remaining suitable for on-device deployment, supporting privacy-preserving and real-time mental health monitoring. These findings highlight the potential of compact, personalised AI systems for scalable digital mental health applications.