Interactive Quality and Reliability Management Using LLMs

SUMMARY & CONCLUSIONSThe potential and hype around Large Language Models (LLMs) are enormous, but to fully realize their benefits, we need to work through the details of use cases and applications. This paper highlights the numerous considerations for designing and building an LLM application around user queries regarding quality and reliability management documentation. In general, out-of-the-box LLMs are often insufficient for the task, the responses to user queries are both incomplete, because the model misses important information or the responses are not detailed enough, and inaccurate, as the model tends to conflate topics and provide information that is not relevant. However, by tweaking the multiple factors that comprise an LLM application, we continue to see improvements in the system.