Psychometric properties of the Clinical Sustainability Assessment Tool (CSAT) short form across three research centers evaluating effectiveness and implementation of a cancer symptom surveillance and management intervention

Abstract Objective The Clinical Sustainability Assessment Tool (CSAT) is designed to capture determinants of sustainable clinical practices over time. Although the full 49-item CSAT instrument has demonstrated strong psychometric properties, the 21-item short form has had limited evaluation. This study aimed to assess the CSAT short form (CSAT Short) across different respondent characteristics and care delivery settings. Methods We evaluated the CSAT Short in a sample of healthcare personnel ( N = 256 respondents) drawn from across three hybrid effectiveness-implementation studies in a research consortium, all of which tested routine symptom surveillance and integration of symptom management interventions in ambulatory oncology care settings in the US. Confirmatory factor analyses (CFA) and mIRT were conducted to assess the CSAT Short's fit to the hypothesized factor structure. Multiple-group CFA was used to test for measurement invariance across groups of respondents with different professional roles, years in current role, and different work settings. Results The hypothesized seven factor structure of the CSAT Short exhibited good fit to the data and strong internal consistency in our sample of healthcare personnel drawn from across three large pragmatic trials (CFI = .99,TLI = .98, X 2 (182) = 658.99, p < .001;SRMR = .031,RMSEA = .10). Tests of measurement invariance indicated the respondent’s role in the clinical setting (i.e., clinician vs. non-clinician) and years in current role (< 10 years vs. ≥10 years) were invariant. However, significant variance was found between respondents from three different Research Centers within the IMPACT consortium. The second-order mIRT model demonstrated acceptable fit based on most indices (M2(56) = 148.69, p < .001; RMSEA = 0.059, 90% CI[0.048, 0.071];SRMSR = 0.057; CFI = 0.917), though the TLI (0.845) was below the recommended threshold. Item-level fit varied, with RMSEA S-X 2 values indicating six items had acceptable fit, nine items had marginal fit, and five items had poor fit. Conclusions The CSAT Short is recommended to assess sustainability in oncology settings, though users should be cautious when comparing scores across different healthcare systems. Tests of invariance were nonsignificant except for variance by Research Center. Despite some items exhibiting suboptimal fit in mIRT, the overall model fit and reliability were strong. This study advances our understanding of sustainability measurement and the applicability of the CSAT Short across implementation settings and respondents.