Word Embedding: The Geometry of Meaning

Read full text Summarise with VIRAT
Authors: Partha Majumdar
Cited by: 0

Summary

No summary yet. VIRAT can read this paper and write a plain-language summary of its aim, methods, key findings and limitations.

Summarise with VIRAT Free to try with your Science Catalogs account.

Abstract

This presentation provides a comprehensive introduction to how modern natural language processing (NLP) systems transform text into meaningful mathematical representations. It begins by examining traditional approaches such as one-hot encoding, highlighting their limitations, including high dimensionality, sparsity, and the inability to capture semantic relationships. The concept of orthogonality is introduced to explain why such representations fail to encode meaning effectively. The presentation then introduces TF-IDF as an improved statistical method that balances term frequency and document rarity to produce weighted word representations. Through structured examples, it demonstrates how TF-IDF enables document comparison and similarity measurement, while also acknowledging its limitations in capturing deeper semantic structures. Building on these foundations, the presentation transitions to word embeddings, where meaning is represented as vectors in a continuous geometric space. It explains how semantic relationships can be modelled through vector arithmetic, illustrated by analogies such as “King − Man + Woman = Queen.” The concept of directional transformations is explored, emphasising that semantic shifts are consistent and interpretable within a structured embedding space. Cosine similarity is presented as the central mechanism for evaluating semantic alignment, focusing on directional similarity rather than magnitude. The presentation highlights how cosine similarity enables analogy resolution and supports semantic reasoning by identifying the closest matching vectors. Finally, the presentation explores higher-level geometric interpretations, including semantic clustering, manifold structures, and outlier detection. These concepts demonstrate how embeddings can organise words into meaningful groups and reveal underlying linguistic patterns. Overall, the work provides a clear, visual, and mathematical framework for understanding how machines interpret language through geometry and vector-based representations.

Fields of Study & Subjects

Topic Modeling Natural Language Processing Techniques Sentiment Analysis and Opinion Mining

Publication details

Journal / source
Zenodo (CERN European Organization for Nuclear Research)
Publication date
23 March 2026
Language
EN