The computational reproducibility of articles published under the Open Data + FAIR policy of IJGIS

Academic journals increasingly require data and code sharing to improve computational reproducibility, but the effectiveness of these policies remains unclear. This study evaluates the changes in reproducibility after the Open Data + FAIR policy was implemented by the International Journal of Geographic Information Science in August 2019 through a systematic audit of 351 articles published between 2020–2024, with a comparison made to 32 articles from the pre-policy period. Using a five-star reproducibility framework based on FAIR principles, we assessed data and code availability, metadata quality, and adherence to open science standards. Results show significant improvement in material availability post-policy, with most articles now including data and code compared to minimal sharing pre-policy. However, computational reproducibility likely remains limited, with most articles achieving only 1–2 star ratings due to inadequate metadata, unclear workflow documentation, and missing details about computational environments. While compliance with basic sharing requirements increased after policy introduction, it is unclear if it has facilitated the comprehensive documentation necessary for independent reproduction. These findings suggest that journal policies focused solely on availability may be insufficient for achieving computational reproducibility in GIScience research, highlighting the need for enhanced standards addressing metadata, workflow documentation, and computational environments.