ReproQuorum : Signed, Scope Deterministic Pipelines and Benchmark Quorums for Verifiable ML Reproducibility

View at publisher
Authors: Surya Pavan Kumar Gudla, Preeti Nutipalli, Chalapathi Rao T
Cited by: 4

Abstract

Reproducing machine learning results across teams and hardware often fails due to hidden randomness, drifting datasets, and unstable environments that alter metrics without visible changes to code or intent. ReproQuorum virtualizes randomness with scoped, counter‑based generators, derives hash‑locked master seeds from commit, dataset fingerprints, and canonicalized configuration, emits signed JSON‑LD provenance receipts, and packages a Nix‑pinned OCI capsule for deterministic re‑execution. Repository continuous integration targets a benchmark quorum of ≥ 2 public datasets per task; the reported experiments enforce ≥ 1 dataset per task. Across named entity recognition (WNUT 2017) and extractive question answering (SQuAD v1.1), replication attains Tier‑A on matched architectures (byte‑identical) and Tier‑B across CPU↔CUDA (metric‑identical with Δ≤10^(-6)). Performance improves by +1.1 F1 on WNUT and +0.7 F1 / +0.6 EM on SQuAD under equal budgets. Capsules and receipts enable one‑command verification and reduce ambiguity about seeds, data, and environments. Code, data manifests, checkpoints, capsules, and model cards are released with signed receipts and container digests to support independent validation.

Fields of Study & Subjects

Scientific Computing and Data Management Machine Learning in Materials Science Software System Performance and Reliability

Publication details

Journal / source
International Journal of Computational Science and Engineering Research
Publisher
Jack Sparrow Publishers
Volume
3
Issue
1
Pages
52–52
Publication date
19 January 2026