Psychometric Evaluation of the Homework Rating Scale—Mobile Application: Reliability and Predictive Validity for Assessing Engagement in Smartphone‐Delivered Cognitive Behavioural Therapy

Self-guided digital mental health apps often show high attrition, highlighting the need to understand engagement beyond behavioural metrics. The Homework Rating Scale-Mobile Application (HRS-MA) was developed to assess cognitive, emotional and attitudinal aspects of engagement. This study evaluated its psychometric properties. The HRS-MA was integrated into the MoodMission app and administered after 30 days of use, alongside additional self-report measures. Data from 2100 users were analysed for reliability, construct, convergent, divergent and predictive validity. The HRS-MA showed excellent internal consistency (Cronbach's α = 0.90; McDonald's ω = 0.94). Factor analyses supported a two-factor structure-engagement and difficulty and obstacles. The measure demonstrated strong convergent and divergent validity and modest predictive validity. A derived coefficient of psychological effort, representing the interaction between engagement and perceived barriers, significantly predicted symptom reduction, particularly in depression. The HRS-MA is a psychometrically sound tool for capturing cognitive and attitudinal dimensions of engagement in digital interventions. It complements behavioural indicators by offering a more comprehensive view of user experience and effort, providing researchers and developers with a valuable framework for optimizing mental health app design and evaluation.