Artifact for ISSTA Submission #1270

This artifact contains the scripts and data necessary to reproduce the figures, tables, and statistical analysis results presented in the paper. Directory Structure analyze_final_results.py: Script to analyze the final classification results (Symptoms, Root Causes) and generate the correlation heatmap. calculate_stats.py: Script to calculate general statistics about the dataset, including reproducibility and Verilog construct prevalence. plot_loc_dist.py: Script to generate the Lines of Code (LOC) distribution plot. final_results.csv: The finalized dataset containing Issue IDs, Consensus Symptoms, and Root Causes. verilator_empirical_analysis.csv: Detailed analysis data including construct counts and complexity metrics for each snippet. verilator_snippets_info.json: Metadata about extracted code snippets (Verilog, C++, Configs) used for reproducibility stats. requirements.txt: Python dependencies. run_all.sh: Shell script to install dependencies and run all analyses in sequence. Prerequisites Python 3.8 or higher pip (Python package installer) Installation & Usage 1. Install Dependencies: It is recommended to use a virtual environment. bash python3 -m venv venv source venv/bin/activate pip install -r requirements.txt 2. Run Replication Scripts: You can run the provided shell script to execute all analyses: bash ./run_all.sh Or run individual scripts: * To generate Figure of LOC Distribution: bash python3 plot_loc_dist.py Output: `loc_distribution.pdf` * To generate Tables of Code Structure: bash python3 calculate_stats.py Output: Console output formatted as text tables and LaTeX rows. * To generate the Heatmap: bash python3 analyze_final_results.py Output: `correlation_heatmap.pdf` and console output. Data Description * final_results.csv: * Issue ID: Unique identifier for the GitHub issue. * Consensus Symptom: The agreed-upon symptom category (e.g., Crash, Miscompilation). * Consensus RC: The agreed-upon root cause category (e.g., Optimization, Scheduling & Emission). * verilator_empirical_analysis.csv: * Contains metrics like loc (Lines of Code), max_nesting_depth, ast_node_count, and counts for specific Verilog constructs (e.g., num_always_comb). Notes The scripts assume they are run from within this directory. Some minor visual differences in plots (fonts across systems) may occur, but the data distribution will be identical.