Analysis of Tokenization Techniques in the Context of Morphology for Multilingual Inputs
Abstract
The process of tokenization entails dividing content into fragments known as signs that can be individual letters, phrases, or subwords.Device transcription, recognition of names, and synthesis of content are all significantly impacted by tokenization.While used for fluent data use, tokenization tasks exhibit intricate problems because of shape parts.A thorough analysis of numerous
Fields of Study & Subjects
Natural Language Processing Techniques
Topic Modeling
Handwritten Text Recognition Techniques
Publication details
- Journal / source
- Engineered Science
- Publication date
- 1 January 2026
- Language
- EN