Reproducibility of Experiments in Networking

Reproducibility is a cornerstone of good scientific practice, enabling validation of experimental results and fostering progress by building upon prior work. The importance of reproducibility in networking research has gained traction in recent years. Notably, the Dagstuhl beginner’s guide to reproducible experimental networking research has provided valuable best practices for documentation, data handling, and experiment design. However, the identification of specific networking components and aspects that prevent experiment reproducibility has not been investigated yet. This work addresses this gap by systematically identifying such components, constructing a taxonomy based on the findings, and proposing guidelines to increase experiment reproducibility. For this purpose, we discuss and analyze several networking use cases: 5G measurement experiments, Internet measurements, AI/ML-based networking experiments, and subjective QoE studies. Through targeted experiments and a review of existing studies, we identify some key sources of variability and discuss strategies to reduce variability and thus increase reproducibility. Our analysis highlights the importance of clearly defining research questions and expected outcomes as they form the foundation for assessing reproducibility. Additionally, our findings show the need for documenting potential sources of randomness, e.g., platform-specific effects introduced by operating systems, prior to experimental design. Minimizing unnecessary variability, e.g., by abstracting or replacing such components, further enhances reproducibility. While not all variability can be controlled due to the inherent complexity of networked systems, the proposed guidelines offer practical steps toward more reproducible networking experiments.