Building Trustworthy AI for Enterprise Support: An Empirical Study of a RAG-Based Architectural Framework

The paper considers an approach to engineering a trustworthy enterprise support service based on a Retrieval-Augmented Generation (RAG) architecture in a high-load ticket-handling environment. The study’s relevance stems from the widespread deployment of generative solutions in contact centers and users’ growing sensitivity to hallucinations, stale data, and unpredictable system behavior. The objective of the research is to obtain an empirical assessment of a full-scale RAG architecture for internal support and to identify engineering decisions that critically determine its reliability. Across five experimental series, empirical data were obtained for key metrics: MRR@10 for retrieval quality, overall refusal and correct-refusal rates, the proportion of confident yet factually incorrect answers, p50/p90/p99 end-to-end latency, the time required to incorporate a document into the index, and the share of interactions involving outdated information. The scientific contribution of the work lies in analyzing a RAG system as an integrated engineering artifact studied on a real-world corporate corpus. It is shown that semantic document segmentation yields a substantial improvement in retrieval quality over fixed-size chunking. That dense semantic search dramatically outperforms sparse keyword-based search, including in terms of the system’s ability to refuse correctly under knowledge scarcity and to reduce dangerous hallucinations. It is established that the generative component is the dominant source of latency. In contrast, a hybrid indexing strategy that combines streaming delta-indexing of critical documents with nightly re-indexing of the entire corpus enables maintaining both knowledge freshness and operational resilience. The paper is intended for researchers and practitioners designing scalable and trustworthy enterprise support systems built on large language models.