Jackknife Ensemble Mutual Information (JEMI)

We present JEMI (Jackknife Ensemble Mutual Information), a mutual information estimator I(X;Y) between continuous features X∈RD and discrete labels Y∈{1,...,K}.JEMI combines three established estimators: Ross (2014), Kraskov (2004), and Kernel Density Estimation, using inverse variance weights empirically derived from 60 OpenML datasets. The optimal weights are rewRoss = 0.028, wKraskov = 0.060, and wKDE = 0.912.We empirically verified consistency across 12 configurations with 5 seed values each, demonstrating a 12-fold reduction in the confidence interval width from N = 100 to N = 800. JEMI achieves a correlation of ρ = 0.987 with the independent Ross estimate. Applied to the Law of Semantic Tolerance, JEMI achieves a success rate of 95.39% across 200 datasets. We also provide an honest analysis of the limitations, including high bias in high-dimensional configurations (D > 10) and redundancy between Kraskov and KDE components (ρ = 0.95).