Transforming written assessment design to embrace AI: what needs to be changed to encourage higher-order critical thinking

Abstract Uncritical use of generative AI (GenAI) responses is a major concern among educators as it can hinder knowledge development, creativity, critical thinking and academic misconduct. To mitigate these repercussions, current discussions predominantly focus on changing assessment methods or policing students’ use of GenAI, which greatly shapes students’ GenAI use, for better or for worse. Few studies have examined how different assessment designs impact the way students use GenAI for coursework and the quality of GenAI-assisted writing. This study uncovered the relationships between assessment design and critical thinking in students’ writing through analysing lecturer feedback on 51 postgraduate ChatGPT-assisted students’ assignments across fourteen modules and assessment information related to the assignments. Results revealed that word limits, genres, information about organisational structures, and cognitive domains required by assessments significantly determined students’ critical thinking performance in their disciplinary writing. Based on the results, we suggested (a) setting word limits based on task complexity rather than module credits, (b) designing integrated tasks with varied assessment methods to encourage critical thinking and knowledge development, (c) providing an appropriate amount of structural information to create space for critical thinking and (d) explicitly signalising cognitive domains required by assessments to address GenAI’s impact on writing. We further encourage educators to critically reflect on the existing assessment guidance and practices to design assessments that cultivate critical AI users in an AI-empowered world.