From Discrete Tokens to Continuous Meaning: The Geometry of Language

Read full text Summarise with VIRAT
Authors: Partha Majumdar
Cited by: 0

Summary

No summary yet. VIRAT can read this paper and write a plain-language summary of its aim, methods, key findings and limitations.

Summarise with VIRAT Free to try with your Science Catalogs account.

Abstract

This paper analyses the evolution of text representation in Natural Language Processing (NLP), charting the transition from discrete symbolic systems to continuous geometric models of meaning. It begins by examining the fundamental limitations of classical, count-based methods such as one-hot encoding and Term Frequency-Inverse Document Frequency (TF-IDF). While foundational, these approaches produce sparse, high-dimensional vectors that treat words as isolated entities, failing to capture the rich semantic and syntactic relationships inherent in human language. This results in computational inefficiency and an inability to perform nuanced linguistic tasks. The analysis then introduces the paradigm shift brought about by word embeddings, which operate on the distributional hypothesis that a word's meaning is derived from its context. By mapping words into a dense, low-dimensional continuous vector space, embeddings position semantically related words in close proximity. This geometric arrangement enables complex linguistic phenomena to be modelled using vector arithmetic, famously demonstrated by the analogical reasoning task `vector(“king”) – vector(“man”) + vector(”woman”) ≈ vector(”queen”)`. Such operations reveal that abstract concepts such as gender, age, or royalty correspond to consistent linear directions in the space. The paper further explains how cosine similarity measures semantic proximity by computing the angle between vectors, providing a robust metric for quantifying word alignment. This geometric framework not only facilitates analogical reasoning but also supports applications like semantic clustering and outlier detection, treating the vocabulary as a highly structured manifold of meaning. The conclusion posits that this transformation of language into a geometric structure, in which meaning is encoded as direction and position, is the foundational principle of modern computational linguistics.

Fields of Study & Subjects

Topic Modeling Natural Language Processing Techniques Sentiment Analysis and Opinion Mining

Publication details

Journal / source
Zenodo (CERN European Organization for Nuclear Research)
Publication date
25 March 2026
Language
EN
Document type
Book chapter