Constitutional Geometry: Enforcing Structural Invariance in Neural Representations via Random Subspace Projection

Read full text Summarise with VIRAT
Authors: Tushar Haldar
Cited by: 0

Summary

No summary yet. VIRAT can read this paper and write a plain-language summary of its aim, methods, key findings and limitations.

Summarise with VIRAT Free to try with your Science Catalogs account.

Abstract

Which training objectives actually induce representation-level invariance in neural networks? This paper provides a sharp negative answer for two widely used approaches: KL-based consistency losses (including Virtual Adversarial Training) fail to reduce representation-space invariance error and can increase it by up to 67% relative to a cross-entropy-only baseline. Standard noise augmentation similarly leaves representation geometry unchanged. Only explicit projection-based geometric constraints reliably induce invariance. Across controlled experiments on Digits, MNIST, and Fashion-MNIST (MLP and CNN architectures, 5 random seeds), a simple fixed random projector P — motivated by concentration of measure — achieves a 96.7% reduction in projection-space invariance error and a 58% reduction in representation distance versus baseline. The constitutional constraint further achieves an 8.5pp accuracy gain under finite noise versus only 1.1pp for Jacobian regularization, showing that geometric anchoring and smoothness objectives operate on different axes. Three propositions formalise the theory: (1) sufficiency — if the constitutional loss converges, ΔC is bounded; (2) necessity — for any K-class classifier there exist models with zero output KL but arbitrarily large ΔC, because the (N_H − K)-dimensional null space of the classifier head is unconstrained by output-level losses; and (3) independence — Jacobian-norm regularization and ΔC control are mathematically independent objectives. Invariance induced in a single random subspace transfers to independently sampled projectors and generalises to structured geometric perturbations never seen during training (97.8% ΔC reduction under rotation and translation). Preliminary scale validation on CIFAR-10 / ResNet-18 confirms the directional result holds at standard benchmark scale. Together, these results show that invariance is a geometric property of representation space, not a byproduct of distributional consistency, smoothness, or data augmentation. Keywords: representation learning, invariance, neural networks, random projection, geometric regularization, adversarial robustness Related work: The geometric intuition for this paper builds on the conceptual framework introduced in Haldar (2026), doi:10.5281/zenodo.19203049.

Fields of Study & Subjects

Adversarial Robustness in Machine Learning Machine Learning and Data Classification Domain Adaptation and Few-Shot Learning

Publication details

Journal / source
Zenodo (CERN European Organization for Nuclear Research)
Publication date
26 March 2026
Language
EN
Document type
Preprint