Machine Learning to Analyze Alternating Treatments Graphs

Alternating treatment designs (ATD) allow practitioners to rapidly compare the effectiveness of multiple interventions without needing long baseline phases. ATDs are typically evaluated using visual analysis and effect size analysis. However, past research suggests that different raters may come to different conclusions when analyzing these graphs. Furthermore, agreement between raters does not equate to accuracy. Artificial Intelligence (AI) technologies may improve the replicability of decision-making while performing adequately with smaller datasets. Within AI, Machine Learning (ML) algorithms can learn patterns and make data-driven predictions. These algorithms can quantify complex, intuitive patterns directly from data that would be difficult or impractical to define explicitly with traditional programming rules. This study investigates the use of ML technology to analyze ATD graphs. Specifically, the researchers examined which feature engineering techniques resulted in predictive performance for an ML model trained on simulated or non-simulated data and tested on non-simulated data. The best-performing models achieved classification accuracy above 90% with type 1 error rates below 15%. Deep neural networks (DNNs) led to the highest accuracy in detecting differentiated effects in ATD graphs while minimizing false positives. These results provide initial evidence that embedding DNNs into analysis of ATD single-case graphs could enhance the replicability of data analysis.