Data-Driven Semantic Axes ICA Reveals That Independent Opposition Dimensions in Word Embeddings Do Not Match Human Conceptual Categories

Read full text
Authors: Tiffney Bare
Cited by: 2

Abstract

We apply Independent Component Analysis (ICA) to 190 WordNet antonym difference vectors in a 50-dimensional GloVe space and discover that the statistically independent semantic opposition dimensions in distributional embeddings do not align with the axes that human conceptual intuition predicts. ICA finds specificity (specific/general), causality (cause/effect), directness (explicit/implicit), and change (static/dynamic) as more geometrically independent than the traditional valence (love/hate), moral (good/bad), and truth (true/false) axes. ICA-discovered axes reduce mean axis entanglement from 0.210 to 0.123 (42% improvement in Experiment 23) and maximum pairwise similarity from 0.744 to 0.430. The geometry-first approach reveals a genuinely new working axis: directness (explicit/implicit), which achieves bidirectional top-3 opposition hits that the original design never produced. Experiment 24 confirms that the architecture-level benchmarks (B2, B3, B5, B8b) are stable at 80%, 80%, 70%, and 100%, regardless of axis design; these properties belong to the GloVe embedding geometry, not the axis labeling. The pole-swap antonym detection ceiling, by contrast, persists across both axis designs, confirming it is a property of the operation rather than the axis definition.

Fields of Study & Subjects

Topic Modeling Natural Language Processing Techniques Biomedical Text Mining and Ontologies

Publication details

Journal / source
Zenodo (CERN European Organization for Nuclear Research)
Publication date
15 May 2026
Language
EN
Document type
Preprint