Automated Generation of Complexity-Validated Decision Scenarios Using Large Language Models

This paper introduces and validates an automated pipeline for generating complexity-controlled decision scenarios using large language models (LLMs). Complexity is treated as a latent construct expressed through four established frameworks: Wood (1986), Campbell (1988), Liu & Li (2012), and Sweller (1994). We evaluated 4,238 scenarios across four decision domains (Analytical, Planning, Communication, Problem Solving) and three complexity tiers (Simple, Moderate, Complex). Measurement validation exceeded psychometric standards: inter-LLM agreement was near-perfect (ICC = 0.997; κ = 0.971), test-retest reliability was excellent (ICC = 0.995, no significant drift p = 0.178), and known-groups validity showed massive tier separation (η² = 0.587, all pairwise p < .001). Factor analysis revealed a dominant complexity construct (λ = 0.87–0.96 for three frameworks) with interactivity forming a weaker secondary dimension (λ = 0.34). Model-performance analyses revealed a near-perfect speed–quality trade-off (r = −0.967, p = 0.007): Llama 4 achieved 5.3× higher throughput but failed at complex tiers, while DeepSeek balanced speed, quality, and full spectrum coverage. The validated instrument provides the measurement infrastructure for the Multi-Dimensional Assessment for AI Cognition (MAAC) framework — a process-oriented cognitive assessment system for AI systems. This work is currently under review for publication.