When Can an AI Answer?

This record provides a reproducible, replayable evaluation artifact for the Governance Kernel project (v2.0). The goal is not to win a benchmark leaderboard; it is controlled validation of protocol semantics: “score-only gating vs. feasibility constraints” under an explicit Right-to-Answer (RtA) policy predicate. What’s included: Source code and configs used to run the evaluation. Evaluation outputs (metrics and summaries) for the run: Run ID: ablations_v2_2026_01_27 Per-instance audit bundles that support deterministic replay. A sweep grid over (theta, delta) showing the risk–coverage trade-off. Core evaluation positioning (to preempt tautology reviews): Oracle here means a feasibility oracle computed by the exact solver under the same formal Sufficiency predicate. It is definition-isomorphic: it is not a learned model and not “ground truth of the world.” The experiment isolates the behavioral difference between methods that only threshold scores and methods that enforce structural feasibility constraints (coverage + conflicts + risk budget). Main results snapshot (N=40; deliver_rate / UDR / OAR): direct: 1.000 / 1.000 / 0.000 naive: 0.750 / 0.667 / 0.000 no_conflict: 0.500 / 0.333 / 0.000 no_risk: 0.500 / 0.333 / 0.000 greedy (feasibility): 0.250 / 0.000 / 0.000 anytime (feasibility): 0.250 / 0.000 / 0.000 oracle_rta_rate: 0.250 Subset deliver_rate: answerable_clean: direct 1.000, naive 1.000, no_conflict 1.000, no_risk 1.000, greedy 1.000, anytime 1.000 unanswerable_missing: direct 1.000, naive 0.000, no_conflict 0.000, no_risk 0.000, greedy 0.000, anytime 0.000 conflicting_evidence: direct 1.000, naive 1.000, no_conflict 1.000, no_risk 0.000, greedy 0.000, anytime 0.000 adversarial_inducement: direct 1.000, naive 1.000, no_conflict 0.000, no_risk 1.000, greedy 0.000, anytime 0.000 Auditability: All generated bundles replay successfully (bundles=40, failures=0). Replay recomputes the commitment root and checks canonicalization consistency, so a replay pass implies “same decision for the same cryptographically committed reasons.” Sweep grid summary (coverage_rate, avg_risk): theta_6000: delta_3 (0.250, 1.000), delta_5 (0.250, 1.000), delta_10 (0.500, 4.450) theta_8000: delta_3 (0.250, 1.000), delta_5 (0.250, 1.000), delta_10 (0.500, 4.450) theta_9000: delta_3 (0.250, 1.900), delta_5 (0.250, 1.900), delta_10 (0.450, 4.944) Implementation note on units (important for reviewers): Support signals are represented as fixed-point integers (scale = 10,000). Thresholds theta are defined in the same scale. Example: theta=8000 corresponds to 0.8000 in normalized units. This avoids floating-point nondeterminism in audit replay. Minimal reproduction steps: Run the evaluation: make eval CONFIG=experiments/configs/ablations_v2.yaml OUTPUT_ROOT=experiments/outputs Replay all bundles (expected failures=0): for b in experiments/outputs/ablations_v2_2026_01_27/bundles/*; do rta-kernel replay –bundle “$b” –verbose || exit 1 done If you get a different outcome, either your environment differs from the recorded artifacts or you found a bug. In both cases, congratulations: the kernel will make it painfully obvious, which is the whole point.