Reliability of a child self-regulation measure within intensive longitudinal designs: MCFA and DSEM approaches

Computing reliability depends on the data set structure. In intensive longitudinal designs (ILDs), reliability for within- and between-person variation (level-specific reliability) must be computed; however, most multilevel studies fail to report reliability at both levels. Commonly used estimates of reliability do not account for the temporal dependency between subsequent measurement occasions (i.e. autocorrelation) or individual differences in measurement properties including person-specific factor loadings or within-person residual variance (i.e. random innovation variance). We illustrate how to compute level-specific and person-specific reliability while addressing multilevel measurement invariance. We analyzed ILD data on preschool-age children’s daily self-regulation, collected from 39 parents up to 100 days (observations = 3359). Using multilevel confirmatory factor analysis (MCFA) and dynamic structural equation modeling (DSEM) with fixed and random effects, we computed level-specific reliabilities. Person-specific reliabilities were derived from the random effect DSEM model. This work demonstrates how MCFA and DSEM meets challenges to understanding reliability across levels and people.