Rigorous Viability Assessment of Machine Learning Projects

Abstract Machine learning offers significant potential for organizations, yet transitioning models from development to deployment remains challenging. Frameworks such as CRISP-ML(Q) and MLOps emphasize the need to integrate business, economic, and machine learning perspectives. However, a systematic literature review reveals a lack of methods that link machine learning perspectives with business objectives. To address this gap, the authors introduce a metric – called profit-per-decision (ppd) – for binary classification that incorporates both model performance and economic impacts. Further, the Viability Assessment Framework is proposed, which utilizes the metric and enables organizations to assess viability at different project stages: pre-development, post-development, and post-deployment. The authors evaluate the framework through expert interviews and a scenario-based evaluation with experts from eleven different companies and develop an open-source web application to support interaction during the case studies. Results confirm the framework’s effectiveness in bridging technical and business perspectives, highlighting its industry relevance.