Dimensional Folding and Distance-Reduction Theory for Similarity Search

Read full text
Authors: Christian Kilpatrick
Cited by: 0

Abstract

This preprint frames dimensional folding and related distance-reduction mechanisms as a working theory to share and explore—not as settled universal fact. When structure-preserving maps (or cheaper approximate metrics) keep the invariants that neighbor search cares about, query cost can fall without destroying retrieval quality; when those invariants fail, measured latency may still drop while Recall@k collapses. One constructive continuous instance is linear PCA-style projection ℝⁿ→ℝᵐ (centerpiece n=74, m=19). On strong-spike synthetic records (four laboratory runs), that fold yields ρ≈0.982, pairwise distance correlation r_d≈1.00, Recall@10≈0.970, and measured brute speedup S_brute≈2.12× against n/m=3.89. An isotropic control at the same 74→19 matches S yet yields Recall@10≈0.07. For continuous databases of stored vectors, that measured path—fidelity plus S∼2×, operated as prefilter–refine—is the constructive solution. To realize the quadrillion-scale ratio R=2^{n−m} (at 74→19: R=2^{55}≈3.60×10^{16}), move into the discrete cardinality regime: enumerate a state space Ω∼2ⁿ and apply a predicate-preserving fold to ∼2ᵐ (Combinatorial Folding Conjecture). An exhaustive bitstring enumeration experiment confirms R=2^{n−m} exactly when the fold preserves the query predicate, and fails when it does not. Continuous stacks need independent multiplicative factors—candidate reduction × cheaper distance × early exit—to approach 10^{15}–10^{16}; 74→19 alone is not that path. Manuscript 2.2.0 (theory edition). Zenodo deposit version 2.0.0. Provisional theory for sharing and testing; laboratory numbers are corroborating instances under stated regimes, not universal proof.

Fields of Study & Subjects

Advanced Database Systems and Queries Data Management and Algorithms Algorithms and Data Compression

Publication details

Journal / source
Zenodo (CERN European Organization for Nuclear Research)
Publication date
13 August 2026
Language
EN
Document type
Preprint