Between delegation and responsibility: an exploratory case study of graduate educators’ conceptualizations of AI-supported assessment using the AI assessment scale

This study investigates how graduate educators conceptualize and apply the AI Assessment Scale (AIAS) within their assessment design practice. Drawing on an exploratory qualitative case study design, the study analyzed eight AI-integrated lesson plans produced by in-service teachers in a technology elective course, supplemented by semi-structured interviews with four participants. Findings suggest that AIAS level selection is experienced not as a technical classification but as an ethical boundary-setting practice, a judgment about delegating evaluative responsibility between human and AI agents. Participants demonstrated variably developed AI assessment literacy: procedural ethics (integrity, authorship) and experiential ethics (learner agency) were more fully articulated than structural ethics (algorithmic bias, data governance). Constructive alignment was strongest when AI was constitutively embedded in the design; conversely, AI integration could shift assessment criteria from content-driven toward procedurally driven evaluation, a drift originating at the design-imagining stage before any tool was deployed. Notably, designing without AI tools heightened participants’ awareness of habitual AI dependence and, in several cases, increased confidence in unassisted design, suggesting that AI assessment literacy may require experiential constraint as well as conceptual instruction. Implications are discussed for teacher education, AIAS professional development, and AI assessment literacy.