Auditing learned criterion weights in technology portfolio screening
Abstract
Overview This release provides the fully revised and reproducible analytical package for the study: Learning outcome-conditioned criterion importance for technology portfolio screening The study has been repositioned from a purported new AI-based weighting method to a leakage-aware protocol for deriving outcome-conditioned proxy-importance profiles and examining their propagation through multi-criteria project rankings. Major changes Replaced the single fitted Random Forest analysis with Elastic Net, Random Forest, Extra Trees and histogram gradient boosting. Implemented five-times-repeated five-fold cross-validation using identical splits across learners. Replaced in-sample impurity importance with held-out permutation importance. Added explicit handling and reporting of negative permutation-importance estimates. Added a secondary target-normalised attainment outcome. Added leakage, programme-grouped and chronological generalisation diagnostics. Corrected the WASPAS and MARCOS implementations. Added conventional comparison profiles and learner/outcome agreement analyses. Added 5,000 bounded weight-perturbation simulations and rank-acceptability diagnostics. Added a frozen data dictionary, pinned environments, deterministic run manifest and GitHub Actions reproducibility workflow. Rebuilt the manuscript around a retrospective benchmarking contribution without causal, preference-learning or prospective deployment claims. Reproduction python -m pip install -r requirements.txt python scripts/run_analysis.py
Fields of Study & Subjects
Multi-Criteria Decision Making
Technology Assessment and Management
Decision Support System Applications
Publication details
- Journal / source
- Zenodo (CERN European Organization for Nuclear Research)
- Publication date
- 8 March 2026
- Language
- EN