Domain-specific dialogue generation and evaluation framework for the new energy industry based on expert knowledge modelling and large language models

In recent years, domain-specific dialogue systems have shown strong potential for industrial use, but many existing models struggle with professional accuracy and deep contextual understanding, particularly in knowledge-intensive fields such as the new energy industry. To address this limitation, we propose a domain-oriented dialogue generation and evaluation framework that integrates expert knowledge modelling with large pre-trained language models. First, a domain-specific knowledge graph is constructed from technical standards, policy documents, and expert reports, with entities and relationships embedded using graph representation methods such as TransE and GraphSAGE. We then develop a dialogue generation module based on LLaMA and ChatGLM, incorporating domain terminology adaptation and knowledge injection to improve response accuracy. A hybrid strategy combining semantic segmentation, dialogue state tracking, and context-aware memory networks is introduced to support robust multi-turn dialogue management. Furthermore, a multi-dimensional evaluation framework is designed, including automatic metrics (BLEU, BERTScore, ProMatch, and knowledge recall) and human evaluation criteria (fluency, accuracy, and domain relevance). Experiments on two curated datasets demonstrate clear improvements over baseline methods, achieving a 5.2% F1 gain in named entity recognition and reducing manual document processing time by more than 80%.