An Intelligent Framework for Learnability Prediction Using Enhanced Cognitive Walkthroughs and Machine Learning

Machine learning (ML) has been increasingly applied in usability engineering to evaluate its dimensions, such as efficiency, errors, and satisfaction. However, the learnability of interfaces—an essential aspect of usability—remains underexplored in ML-driven evaluations. This paper presents a predictive framework that refines traditional Cognitive Walkthrough (CW) questions through semantic alignment with learnability factors and integrates expert evaluation outcomes with ML models. Validation proceeded in two stages. In the first, a case study on Microsoft OneNote with predefined tasks assessed the effectiveness of the Enhanced Cognitive Walkthrough (ECW) questions. Results showed that the ECW questions provided clearer and more actionable insights into learnability issues compared to the traditional CW formulation. In the second stage, ECW evaluation outputs were paired with real user feedback to create ground-truth labels for task learnability. Using these user-based labels, three ML models were trained. Results showed moderate predictive performance (Macro-F1 = 0.41–0.55; Balanced Accuracy = 0.55–0.61). Cross-interface testing between OneNote and StarUML showed partial generalization, suggesting that ECW-based expert evaluations can support predicting learnability across interfaces when combined with actual user perception. These results represent a preliminary feasibility demonstration rather than a fully validated predictive model, demonstrate that combining ECW with targeted user feedback provides developers and UX practitioners a clearer, more reliable, and data-driven method to identify and prioritize learnability issues early in the design process.