Residual Mapping Audit (RMA) v3 — A Framework for Model-Class Dependent Residual Analysis in Parametric Systems
Abstract
The Residual Mapping Audit (RMA) is a framework for quantifying explanatory mismatch in parametric systems. Given a system Y = f(X) and an explanatory model g(X), RMA evaluates whether residuals R = Y − g(X) retain predictive structure evidence that the mismatch stems from the limited expressivity of the chosen model class, not from intrinsic complexity in the system itself. This release (v3) does not change the core RMA method from v1/v2 (coverage, residual R², IBDS, TIR). It adds a statistical robustness layer: seven checks addressing specific failure modes a residual-mapping audit can fall into when used naively false positives from small samples, deterministic artifacts of the operator/pipeline, redundant features inflating coverage, composite targets that cancel signal, and order-dependence. New in v3: permutation testing, invariance testing, an imposed-structure warning in classify(), composite-target detection with automatic component re-analysis, LOO-CV + bootstrap confidence intervals, VIF (multicollinearity) reporting, and order-stability testing. All additions are opt-in and backward compatible default behavior reproduces v2 output exactly (verified against the reference implementation). Two bugs were found and fixed during development: a NaN in residual_r2under Leave-One-Out CV (single-row test folds have no intra-fold variance to compute R² from), and a Benjamini-Hochberg FDR q-value bug incombinatorial_search where a disabled code branch (leftover `if False`) allowed q-values above 1.0 and non-monotonic q-values both fixed and covered by regression tests. Engine correctness is validated on synthetic ground-truth data (five scenarios with known, fabricated answers) and formalized as a five-case pytest-style unit suite, each case verified across five independent randomseeds in addition to its base run (39 base assertions + 130 multi-seed assertions, 100% passing, zero failures). Applied validity is separately checked against UCI Energy Efficiency (n=768) against published literature,plus two exploratory worked examples (Boston Housing, n=264; the ARGIRA sonification-of-paintings corpus, n=227 down to n=29 across feature families). This deposit is methodological infrastructure for the ARGIRA research series (see Series concept DOI below), not a new empirical finding about the visual-acoustic mapping the series investigates. **Files included:**- `rma_core_v3.py` — full module: v2 core plus the seven v3 robustness additions (permutation_test, invariance_test, compute_vif, detect_composite_target, bootstrap_residual_r2_ci, order_stability_test, imposed-structure warning in classify()).- `rma_auditor_v3.html` — standalone, 100% offline interactive auditor (Pyodide/WASM, no server, no data leaves the browser). Same engine as `rma_core_v3.py`, embedded byte-identical (verified) as base64 and exec'd at runtime, not a reimplementation. Adds a configurable CV-strategy panel, VIF table, composite-target detection banner, invariance/permutation test panel, and a combinatorial-search panel with a runtime ETA probe and confirmation gate for expensive searches.- `README.md` full documentation: changelog, per-scenario validation tables, threshold-by-threshold audit of classify()'s conventions, and worked examples on real and synthetic data.- `synthetic_rma_test.py` / `synthetic_rma_test_results.json` five-scenario synthetic ground-truth test (engine correctness, not applied validity).- `test_fdr_fix.py` — regression test for the Benjamini-Hochberg q-value fix.- `test_case_perfect_linear.py`, `test_case_pure_noise.py`, `test_case_quadratic_misspec.py`, `test_case_perfect_collinearity.py`, `test_case_composite_target.py` — five-case pytest-style unit suite formalizing the synthetic ground-truth scenarios as named, assert-based regression tests. - `run_multiseed_check.py` — multi-seed robustness runner: reruns each of the five cases across 5 independent seeds (2, 3, 7, 11, 99) beyond it own base-seed run, reusing the existing `test_*` functions unmodified (130/130 assertions passing, zero failures).- `energy_efficiency.csv` — raw UCI Energy Efficiency dataset (n=768;Tsanas & Xifara, 2012). `energy_efficiency_combinatorial_search_max2.csv` — full combinatorial search output (X1–X8 vs. Y1/Y2) on the same dataset, with FDR correction.- `boston_housing_named.csv` — 264-row worked example, first exploratory pass outside the ARGIRA/Energy Efficiency corpora.- `consolidated_paintings.csv` — wide, sparsely-populated join of four feature families from the ARGIRA sonification corpus (n = 227, 72, 41,29 across families), used in the screening-pass worked examples. Full changelog, per-scenario validation tables, threshold-by-threshold audit of classify()'s conventions, and worked examples are in README.md.
Publication details
- Journal / source
- Zenodo (CERN European Organization for Nuclear Research)
- Publication date
- 1 July 2026
- Language
- EN