Bing Cheng and Howell Tong’s contribution to the Discussion of ‘Statistical exploration of the Manifold Hypothesis’ by Whiteley et al.

Due to space limitations, we highlight only two issues with the manifold hypothesis (MH) here: (1) information geometry speaking, MH represents a strong restriction on the data space; (2) in terms of generative statistics/AI, the MH reduces the capacity of a generative model to represent the real world. To address the above issues, among others, we briefly outline below an information geometrical approach. Let χ be a data space. Consider an infinite-dimensional manifold M={f|f(x),x∈χ} of smooth probability density functions. Define the tangent space where ft(x)=γ(t)(x)⁠, γ:(−δ,δ)→M is a smooth curve, t being the parameter such that γ(0)=f. Endow TfM with the Fisher–Rao metric gf at f∈M by It holds that and it is algebraically equivalent to the space of score functions, namely where s(x)=h(x)/f(x)⁠. Define the covariate tangent subspace S of TfM of dimension ≤n by Then, because S is a finite-dimensional closed subspace of TfM⁠, TfM=S⊕S⊥,meaning to every h∈TfM⁠, there exists uniquely the orthogonal decomposition where hs∈S⁠, ε(x)∈S⊥⁠, and w∈Rn⁠. Define the covariate Fisher Information Matrix (cFIM) as Gf=((Gf)ij)n×n⁠, where (Gf)ij=gf(∂f/∂xi,∂f/∂xj)⁠. Let D be a locally parameterized manifold via the smooth embedding x=r(y)⁠, where y∈Rd⁠, the intrinsic coordinates with d