Hierarchical Phrase Compression: A Recursive Dictionary-Based Framework for Structured Text Encoding

This paper introduces Hierarchical Phrase Compression (HPC), a recursive dictionary-based algorithm for compressing structured text by identifying and substituting high-frequency contiguous word sequences with composite tokens. Unlike conventional subword tokenisation methods such as Byte Pair Encoding (BPE), HPC operates at the word and phrase level and permits recursive multi-level substitution. The method produces a hierarchical vocabulary in which composite tokens encode frequently recurring structural units of language. The resulting representation reduces sequence length while preserving structural coherence and enabling efficient search and downstream processing. This paper formalises the algorithm, analyses its compression characteristics, and positions it as a practical pre-processing framework for structured text systems.