Inter- and Intrarater Reliability of Forensic Experts in Child Maltreatment Risk Assessments – An Explorative Vignette Study

Expert recommendations for family courts in child protection cases have serious consequences, yet evidence on their reliability is limited. This exploratory vignette study examined inter- and intrarater reliability of child maltreatment risk assessments by German mental health professionals. Agreement was analyzed cross-sectionally (Nt1 = 32) and longitudinally after three months (Nt2 = 16), including the influence of irrelevant contextual information. Interrater reliability was low (ICCs ≤ .26), and intrarater reliability ranged from low to moderate (κs ≤ .35; ICCs ≤ .61). The findings indicate substantial variability in expert judgments, likely due to insufficient standardization of assessment procedures.