The Principle of Local Coherence A Unified Information-Theoretic Framework for Classification, Regression, and Anomaly Detection Supplementary Document Formal Derivation of the Bacteria Phenomenon

This document provides a complete, rigorous derivation of the Principle of Local Coherence (PCL) and its application to the Bacteria Phenomenon—the empirical observation that machine learning models can generalize even when the local coherence ratio ρ ≥ 1 in the original feature space. We establish three core equivalences with 10/10 mathematical rigor (purely algebraic, noasymptotic assumptions): • Classification: ρ < 1 ⇐⇒ Acc1-NN > pmax (1-NN beats majority baseline) • Regression: ρ < 1 ⇐⇒ σ2 local < σ2 global (local variance is lower) • One-Class: e > 1 ⇐⇒ D > Q1−α (anomaly exceeds empirical quantile)The Bacteria Phenomenon is formally justified by proving that if I(X; Y ) > 0 (features carry information), then there exists either sufficient sample size N or transformation ϕ such that ρ < 1. This connects the PCL to Cover & Hart (1967), Stone (1977), and informationtheoretic foundations via Fano’s inequality. All proofs are included with explicit assumptions and no gaps.