Integrating Machine Learning Standards in Disseminating Machine Learning Research

The increasing use of AI-based approaches such as machine learning (ML) across diverse scientific fields presents challenges for reproducibly disseminating and assessing research. As ML becomes integral to a growing range of computationally intensive applications (e.g., clinical research), there is a critical need for transparent reporting methods to ensure both comprehensibility and the reproducibility of the supporting studies. There are a growing number of standards, checklists, and guidelines enabling more standardised reporting of ML research, but the proliferation and complexity of these make them challenging to use—particularly in assessment and peer review, which has, to date, been an ad hoc process that has struggled to throw light on increasingly complicated computational supporting methods that are otherwise unintelligible to other researchers. Taking the publication process beyond these black boxes, GigaScience Press has experimented with integrating many of these ML standards into the publication process. Having a broad scope necessitated looking at more generalist and automated approaches. Here, we map the current landscape of artificial intelligence (AI) standards and outline our adoption of the Data, Optimization, Model, Evaluation (DOME) recommendations for ML in biology. We developed a publishing workflow that integrates the DOME Data Stewardship Wizard (DOME-DSW) and DOME Registry tools into the peer review and publication process. From this publisher’s case study, we provide journal authors, reviewers, and editors with examples of approaches, workflows, and strategies to more logically disseminate and review ML research. This demonstrates the need for continued dialogue and collaboration among various ML communities to create unified, comprehensive standards and to enhance the credibility, sustainability, and impact of ML-based scientific research.