Supplementary Material: Benchmarking Methodology for Industrial Intrusion Detection

This repository contains the supplementary materials for the manuscript “From Scores to Workload: FAH-Calibrated Benchmarking Methodology for Industrial Intrusion Detection”. The supplement is intended to support reproducibility and transparent auditing of the paper’s methodology and results, with a particular focus on the proposed false-alarms-per-hour (FAH) calibration and benchmarking procedure. Contents Benchmark script used to run all experiments (including FAH calibration and detector evaluation). Windowed feature datasets (Parquet) and run metadata (CSV) required to reproduce the benchmark. Benchmark outputs for the FAH targets reported in the paper (α ∈ {1, 5, 10, 15, 20} FAH), including: summary tables (e.g., achieved FAH, precision/recall/F1, AUROC/AUPRC, latency), per-method metrics and breakdowns (DoS vs FDI), figures (FAH–performance tradeoffs, achieved FAH, attack-type performance, latency boxplots, PR/ROC curves). Configuration and manifests describing preprocessing decisions (feature inclusion/exclusion, scaling families) and runtime options. How to reproduce (quick start) Create a Python environment and install dependencies. Run the benchmark using: python compare_detectors_benchmark_v1.py `--windows all_windows.parquet `--runs runs.csv `--outdir benchmark_fah5 `--treat-pre-attack-as-normal `--train-on baseline_only `--threshold fah `--target-fah 5 `--device cpu `--batch 4000 Outputs are written to an outdir/ folder (e.g., benchmark_fah5) containing machine-readable tables and the corresponding figures. Note The core contribution evaluated here is the FAH-calibrated thresholding methodology, which anchors detector operating points to an interpretable alarm workload (alarms/hour) rather than fixed quantiles, EVT-only tail parameters, or per-window FPR. Due to finite horizons and score tie/plateau effects, some detectors cannot realize certain nominal FAH targets; the benchmark reports achieved FAH as a first-class quantity, consistent with the paper’s claims.