Digital Hieroglyphs: Orthogonal Composite Tokens for Vision-Language-Action Models

Vision-Language-Action (VLA) models for embodied AI face a fundamental representation challenge: text, images, and motor commands exist in incompatible modality spaces that must be "translated" through separate encoders and adapters. Current approaches either serialize all modalities into discrete token sequencesakin to "tying knots on a rope"or render text as images for unified visual processing [14]. We propose a third paradigm: Orthogonal Composite Tokens, which we term Digital Hieroglyphs. Inspired by hieroglyphic scripts that fuse pictorial form and semantic meaning into unified symbols, our framework encodes semantic (language), perceptual (vision), and motor (action) information as orthogonal components within a single high-dimensional vector mathematically analogous to superposing frequencies in a musical chord. We present concrete encoding pipelines for each modality and analyze computational benefits: by compressing multi-token concepts into dense composite representations, we reduce effective sequence length (and thus quadratic attention cost) while exploiting GPU tensor cores through "wide matrix multiplication." This approach enables native cross-modal operationssuch as changing an object's color without regenerating action sequences and offers a principled path toward unified world models for embodied agents.