Large Language Model-Assisted Evolutionary Neural Architecture Search for Cognitive Diagnosis

Neural architecture search (NAS) has made remarkable progress in various field tasks and is also used in fields like cognitive diagnosis (CD) in intelligent education. Due to the optimization challenges of the tree-like and variable-length model topology raised by the search space, existing NAS for CD intuitively uses evolutionary algorithms and empirically develops a genetic operator based on basic tree modifications. Nevertheless, the operator's stochastic nature may impede rapid convergence, as the choice of modifications is random and ignores parent information, thereby failing to generate effective models from most parents. To this end, this paper proposes a novel evolutionary NAS approach for CD based on a tailored genetic operator, where the operator incorporates powerful large language models (LLMs) as its key components for tree-encoded model generation. Specifically, three roles of LLMs are defined in the LLM-assisted operator through structured prompt engineering that explicitly defines the LLM's role, constrains the architecture representation, standardizes the output format, and incorporates Hall-of-Fame guidance. The first LLM focuses on exploitation, enhancing model performance through minor incremental adjustments to existing architectures, the second emphasizes exploration, introducing innovative models through creative combinations that have not been previously considered, and the third aims at balancing the contributions of the first two, ensuring a harmonious integration of exploitation and exploration. In each generation, only one role of LLMs is used through an LLM role-switching strategy. Additionally, the empirically-built genetic operator is also employed for partial models' generation for better diversity. Experiments on several datasets show the performance superiority of models searched by the proposed approach over handcrafted models and automatically-designed models. Besides, the effectiveness of the LLM-assisted operator in accelerating algorithm convergence is validated.