When Specialized Beats General: Embedding-Based vs. Large Language Model Classification for the MEPA Pedagogical Ontology

Accurately classifying user inputs into pedagogical ontologies is essential for conversational Artificial Intelligence (AI) systems in educational technology. The MEPA ontology comprises seven specialised pedagogical concepts designed to represent personal learning experiences, and presents unique classification challenges due to its domain-specific nature and conceptual nuances. This paper evaluates the ability of state-of-the-art large language models to perform this specialised classification task effectively. Multiple LLM variants were tested, including GPT-5 (Standard, Mini, and Nano) and the Claude model families. All were configured with extended reasoning capabilities and optimised prompts. Despite these optimisations, LLMs achieved suboptimal classification performance on our test dataset of 350 expert-labelled samples. In response, we developed a specialised classification system that combines OpenAI’s Text Embedding 3-Large model with a feedforward neural network that was trained on 14,000 balanced samples. Our custom model achieved 94% accuracy, outperforming the best LLM baseline by 12%. We analyse the factors contributing to the limitations of LLMs in highly specialised ontological contexts and discuss the practical implications for educational technology systems that require precise alignment with pedagogical frameworks. Our findings suggest that domain-specific models can significantly outperform general-purpose LLMs for specialised classification tasks.