A Principled Estimator of Intra-Cluster Correlation for the Stratified Causal Discovery Threshold

Cluster-aware causal discovery reduces sample complexity from <i>O</i> (<i>d </i>3) to&nbsp;<i>O</i> (<i>d</i>&nbsp;log<i> d</i>) through stratified gradient estimation, but the decision to stratify depends on a critical threshold <i>n</i> crit whose practical computation requires an intra-cluster correlation parameter <i>p</i>. Current practice sets&nbsp;<i>p</i>&nbsp;= 0.3-the median within-cluster Pearson correlation across cancer types-a convention with no theoretical basis that shifts&nbsp;<i>n</i>&nbsp;crit by several hundred samples at typical dimensions (<i>d </i>= 150). We propose ˆ︁&nbsp;<i>p</i>, a gradient-based estimator that exploits the geometric structure of NOTEARS score matrices. We prove <i>V n</i>-consistency via matrix concentration, derive its asymptotic distribution, and bound the resulting&nbsp;<i>n</i>&nbsp;crit decision error. On the Sachs protein signaling benchmark-real single-cell data with 17 known causal edges-ˆ︁&nbsp;<i>p</i>&nbsp;converges to a small but non-zero score-space correlation with bias decaying from 0.0034 at <i>n </i>= 200 to 0.0001 at <i>n</i> = 853. On the TCGA pan-cancer benchmark, ˆ︁ <i>p </i>correctly returns near-zero values when pilot NOTEARS cannot recover structure, triggering a conservative default that prevents erroneous clustering recommendations. We prove that when <i>p S</i> = 0, the estimator automatically tightens <i>n</i> crit to its theoretical minimum-a behavior that is not an estimation failure but the correct statistical decision, since any potential benefit from clustering in this regime is dominated by the additional noise it introduces. Together, the theoretical guarantees and multi-platform validation establish ˆ︁&nbsp;<i>p</i>&nbsp;as the first principled alternative to the ad-hoc&nbsp;<i>p</i>&nbsp;= 0.3 convention, restructuring the decision logic of stratified causal discovery.