AI leads, humans lead, or collaborate? Empirical findings and the SAGE roadmap for embedding GenAI in systems analysis and design education
Abstract
Generative AI orchestration competencies, the abilities to evaluate, adapt, and integrate AI outputs with human judgment, remain inadequately addressed by general AI literacy frameworks. This paper applies and extends the SAGE (Structured AI-Guided Education) framework to systems analysis and design education through three experimental assessments targeting requirements synthesis, formal modelling correction, and design evaluation. In the Brisbane baseline cohort ($n = 13$ groups), 84% demonstrated selective and justified AI use (Balanced Integrator or Selective Adapter), and no group reached expert-level synthesis; the subsequent cross-campus cohorts (n = 5 groups, yielding $N = 18$ template-compliant groups analysed overall out of 21 participating groups) corroborated these patterns. Students consistently recognised accessibility needs when working with user requirements and interface designs, but almost entirely neglected them when producing system architecture diagrams, suggesting that this competency depends on continuous prompting rather than transferring automatically between tasks. When AI was used to generate formal data flow diagrams from the students' specifications, the resulting errors were not random but clustered around system boundary classification, exception handling, and data lifecycle completeness. These findings carry implications for how educators structure AI collaboration in technical curricula, particularly regarding the sequencing of human and AI contributions, the need for stage-specific scaffolding, and the role of explicit justification in developing professional judgment.
Fields of Study & Subjects
Ethics and Social Impacts of AI
Explainable Artificial Intelligence (XAI)
Teaching and Learning Programming
Publication details
- Journal / source
- STEM Education
- Publisher
- American Institute of Mathematical Sciences (AIMS)
- Volume
- 6
- Issue
- 2
- Pages
- 194–229
- Publication date
- 1 January 2026