Replication Package: The State of Open Science in Software Engineering Research: A Case Study of ICSE Artifacts

This replication package accompanies our study evaluating the executability, reproducibility, and usability of replication packages from ICSE research papers. We analyzed 100 artifacts from ICSE papers published between 2015-2024, assessing their practical usability through systematic execution attempts in controlled environments. Key Findings: Only 40% of artifacts were fully executable Only 32.5% ran without any modifications 82.5% required moderate to high effort to execute Among executable artifacts, only 35% reproduced the original results