Measuring High-Priority Outcomes in Autistic Adults: Initial psychometric assessment of the instruments in the AASPIRE Measurement Toolkit

Abstract Background Adult autism services research is hampered by a lack of accessible self-reported outcome measures. The AASPIRE Outcomes Project used a community-based participatory research (CBPR) approach to create and test the AASPIRE Measurement Toolkit, a set of accessible survey instruments for use in real-world settings. The core toolkit contains 12 characteristics modules and 19 outcome measures, each with self-reported and caregiver-reported versions. Methods We used our CBPR-nested Delphi process, our collaborative instrument adaptation/creation process, and cognitive interviews to ensure accessibility and content validity. We then conducted a longitudinal survey to validate the outcome measures in a pragmatic sample of 870 autistic adults from two healthcare systems, two disability service systems, and the larger autistic community in the United States. Participants completed surveys at 3 time points (baseline and follow-ups at 5-11 months and 12-18 months). A 15% random subset completed an additional retest survey. We assessed 1) accessibility using completion rates and perceived ease of use; 2) internal consistency using Cronbach’s alphas; 3) convergent validity using Pearson’s correlations; 4) two-week test-retest reliability using intraclass correlation coefficients; and 5) six-month responsiveness to change by comparing self-perceived change with change in scores. Results Over 90% of participants reported the survey items were easy to understand. The outcome measures and their pre-determined subscales demonstrated strong content validity, internal consistency reliability, test-retest reliability, convergent and discriminant validity, and responsiveness to change. Conclusion The study provides strong evidence supporting the psychometric properties of the direct report versions of 19 outcome measures in the AASPIRE Measurement Toolkit. We will report additional in-depth assessments separately for each measure. Further research is needed to validate the caregiver report versions. All instruments are available for free and can help clinicians, service providers, advocacy organizations, and researchers assess the effectiveness of interventions and follow changes in outcomes over time. Community Brief Why is this an important issue? Autistic adults have worse health and social outcomes than the general population. To learn how to improve outcomes, we need an accurate way to measure them. Most existing survey measures are hard for autistic people to use, not tested with autistic people, or can’t measure if the outcome has changed. What was the purpose of this study? The AASPIRE Measurement Toolkit is a set of accessible questionnaires (also known as “survey instruments”) that can be used to measure the outcomes most important to autistic adults. The purpose of this study was to test how well these survey instruments work. What did the researchers do? We used a community-based participatory research (CBPR) approach, where scientists and autistic community members worked together as equal partners. We collaboratively selected, adapted, and created the instruments in the Measurement Toolkit. We made a version for autistic people and a version for caregivers. The 19 outcome measures focused on quality of life, health outcomes (overall health, flourishing, depression, anxiety, autistic burnout), social outcomes (community participation, self-determination, choices and decisions, employment), and supports and services (social support, barriers to communication, healthcare services, disability services, and adequacy of supports and services). Twelve modules asked for additional information describing participants. We used these 31 instruments with a diverse group of 870 autistic adults and caregivers. They took the survey three times over about a year. We used statistics and feedback to test how well the instruments worked. What were the results and conclusions of the study? The survey measures tested very well: 1) They were accessible. Most people answered all the survey questions and said it was easy to use. 2) They were “internally consistent.” That means that people gave similar responses to different questions about the same idea. 3) They had good “test-retest reliability. ” That means that, as expected, responses didn’t change much over a two-week period. 4) They had good “convergent validity.” That means that they related to each other in expected ways. 5) They were “responsive to change.” That means that changes in scores over 6 months matched participants’ own perceptions of change. Researchers and service providers should use our measures. We feel our measures work well because autistic people were involved at every step. What is new or controversial about these findings? Previously, there weren’t studies to see if measures for these outcomes worked for autistic adults and could detect change. This study furthers what we know about adapting and co-creating accessible, accurate measures for autistic adults using CBPR. What are potential weaknesses in the study? Not enough caregivers took part to test their version of the measures. We think these measures would work for other people too, but we only tested them with autistic adults. How will these findings help autistic adults now or in the future? These measures will help researchers and service providers know if what they are doing to improve outcomes is working. That will help them create services that can do a better job improving autistic peoples’ lives.