Automated Alignment of Math Items to Content Standards in Large‐Scale Assessments Using Language Models

Abstract Accurate alignment of items to content standards is critical for valid score interpretation in large‐scale assessments. This study evaluates three automated paradigms for aligning Math items with four domain labels and nineteen skill labels. First, we extracted instruction‐tuned embeddings, trained a suite of classical supervised machine‐learning models, and examined whether principal component analysis (PCA) improved performance. Second, we fine‐tuned a set of encoder‐only language models (11 BERT‐family models) for domain and skill alignment. Third, we explored ensemble learning via majority voting and stacking with multiple meta‐models. Results show that DeBERTa‐v3‐base achieved the best weighted‐average F 1 score for domain alignment (0.950), whereas RoBERTa‐large achieved the best weighted‐average F 1 score for skill alignment (0.869). PCA substantially improved linear classifiers based on embeddings but did not outperform fine‐tuned encoder‐only language models. Ensemble methods did not surpass the best single encoder‐only language model. Overall, the study provides a systematic comparison of embedding‐based classification, fine‐tuned encoder‐only language models, and ensemble strategies for automated item‐to‐standards alignment.