Non-Semantic Projection and Meaning Phase Transition in Transformer Architectures

Read full text
Authors: Jun Sakai
Cited by: 0

Abstract

This paper argues that semantic meaning does not evolve gradually within Transformer internal states.Instead, Transformer computation consists of continuous non-semantic vector-space operations,while explicit semantic interpretation emerges discontinuously at the output boundary. We propose a four-phase internal dynamic model:(1) temporal structure removal,(2) non-semantic projective attention,(3) complementary absorption of unselected information via residual connections,and (4) a meaning phase transition at the final projection. This perspective provides a structural foundation for pre-semantic safety diagnosticsbased on internal geometry rather than semantic content.

Fields of Study & Subjects

Embodied and Extended Cognition Neurobiology of Language and Bilingualism Natural Language Processing Techniques

Publication details

Journal / source
Zenodo (CERN European Organization for Nuclear Research)
Publication date
1 January 2026
Language
EN
Document type
Preprint