'I Feel 4 out of 5 Depressed'? An Empirically-Supported Argument for Meaningful Measurement of Mental Health

Background: Measurement is considered a&nbsp;cornerstone&nbsp;of&nbsp;evidence-based mental health research and practice.&nbsp;Validated self-report measures&nbsp;require respondents to translate lived experience into&nbsp;standardized&nbsp;numerical scores that are&nbsp;thought to be&nbsp;valid and comparable.&nbsp;But what do people&nbsp;mean when they score, and can&nbsp;researchers and clinicians&nbsp;interpret and use these scores as validly as they assume?&nbsp;Going&nbsp;beyond face validity, qualitative research methods&nbsp;are used to&nbsp;empirically&nbsp;elucidate&nbsp;the&nbsp;meaning-making processes&nbsp;involved in self-report scoring,&nbsp;and&nbsp;to scrutinize whether standardized score interpretation is as valid as assumed.<br><br>Methods: Fifty-three respondents with and without depression completed the Beck Depression Inventory–II. Reflexive thematic analysis was&nbsp;applied to‘real-time’ meaning-making&nbsp;data elicited&nbsp;through a thinking-aloud method (TAM),&nbsp;and to ‘recalled’ meaning-making&nbsp;data from&nbsp;subsequent cognitive interviews (CI).<br><br>Findings: Scoring&nbsp;involved&nbsp;active&nbsp;and individual&nbsp;meaning-making&nbsp;processes.&nbsp;Respondents frequently&nbsp;encountered&nbsp;known psychometric problems, including confusion about item meaning and misfit of response options (Theme 1), and variation in contextual and time frame references (Theme 2). Substantial between- and within-respondent variation was found in&nbsp;how these issues&nbsp;were&nbsp;solved.&nbsp;Consequently,&nbsp;identical scores&nbsp;can reflect markedly different meanings. In selecting solution strategies, respondents demonstrated strong awareness of the anticipated&nbsp;interpretation of&nbsp;their scores and actively attempted to secure valid interpretation by the test administrator (Theme 3).<br><br>Interpretation: These findings challenge two&nbsp;fundamental&nbsp;requirements&nbsp;for&nbsp;psychometric validity:&nbsp;1)&nbsp;respondents&nbsp;do not&nbsp;systematically&nbsp;score as intended, and&nbsp;2) standardized&nbsp;response formats&nbsp;do not&nbsp;ensure&nbsp;consistency&nbsp;within and between respondents. Rather, meaning is continuously shaped in a hermeneutic process between respondent, questionnaire, and administrator.&nbsp;Test administrators&nbsp;should therefore&nbsp;empirically&nbsp;and&nbsp;reflexively&nbsp;evaluate whether score interpretation is valid in local contexts.&nbsp;This study provides an empirical argument to approach mental-health measurement as a&nbsp;meaningful, interpretative practice, and urges to continuously&nbsp;validate&nbsp;data interpretation&nbsp;in the action&nbsp;of measurement.&nbsp;