DIFFICODE: A Dataset and Graph Spectral Framework for Programming MCQ Difficulty Estimation

Estimating the difficulty of multiple-choice questions (MCQs) is a core problem in educational natural language processing, supporting applications such as personalized assessment, curriculum design, and adaptive tutoring. While content-based difficulty estimation for text-centric MCQs is widely explored, programming MCQs remain underexamined due to the lack of domain-specific benchmarks and the absence of models that capture their structural and semantic properties. Unlike linguistic questions, programming MCQs are governed by program execution semantics, control flow, and distractor options that encode alternative interpretations of the same code. Consequently, difficulty arises not from surface text complexity but from structured interactions between the program, the stem, and mutually competing options, where multiple answers may simultaneously appear consistent with the program’s behavior-an ambiguity that cannot be captured by models that treat MCQs as flat token sequences. To address these challenges, we introduce DIFFICODE, a dataset of 2,637 programming MCQs annotated with difficulty levels, and Hierarchical Graph Spectral Aggregation (HGSA) Framework for difficulty estimation. HGSA represents each MCQ as a typed pedagogical graph over the program, stem, and answer options, propagates information across this structure, and applies spectral partitioning to isolate competing semantic configurations, yielding answer-agnostic measures of option competition and ambiguity for difficulty prediction. Extensive experiments with multiple pretrained encoders augmented with HGSA as a post-encoder structural module show consistent improvements in both accuracy and macro-F1. Ablation studies confirm the importance of graph structure, higher-order propagation, and spectral separation for modeling difficulty in programming MCQs.