Replication package for 'The Effect of Complexity and Provenance on Code Review Decisions: Evidence from a Controlled Experiment'

This artifact accompanies the paper “The Effect of Complexity and Provenance on Code Review Decisions: Evidence from a Controlled Experiment” (FSE 2026) and enables full reproduction of the reported results. The package contains all materials required to replicate the study, including the experimental setup, anonymized dataset, and analysis scripts. Contents tool/: Experimental platform used to conduct the study, based on the CRExperiment framework, including the code patches used in the experiment. scripts/: R scripts for data analysis and the anonymized dataset of valid participants. misc/: Supporting materials such as experiment flow diagrams, attention checks, and demographic summaries. Reproducing Results All results can be reproduced using a standard R environment. From the root directory, run: Rscript scripts/analysis.R This script loads the dataset and reproduces the statistical analyses reported in the paper. Demographic analyses and figures can be generated with: Rscript scripts/demographics.R Requirements R (≥ 4.2) for analysis Python 3 (only required for the experimental tool; not needed for reproducing results) No specialized hardware or complex setup is required. Notes The dataset contains only anonymized participant data used in the final analysis. Data from pilot runs are excluded. This artifact provides the complete set of analyses performed in the study, including additional exploratory tests.