Understanding User Requirements in LLM-Augmented BIM Systems: A TAM-Based Evaluation of NADIA-S

Recent research on integrating large language models (LLMs) with building information modeling (BIM) systems has heightened expectations for overcoming technical barriers that hinder BIM adoption. Despite these expectations, there remains a lack of understanding regarding users’ requirements for LLM-augmented BIM systems. Understanding these requirements is crucial for guiding development and addressing adoption barriers, yet this area remains underexplored. This study aims to investigate the factors that could drive the adoption of LLM-augmented BIM systems. An expert evaluation was conducted on an LLM-augmented BIM system—“Natural-language-based Architectural Detailing through Interaction with AI via Speech (NADIA-S)”—which is built upon a six-step, generalized LLM-augmented BIM framework. Through a case study, experts performed interactive design detailing and compliance-checking tasks in a BIM model using NADIA-S. Responses were collected through questionnaires based on the technology acceptance model (TAM) using a 5-point Likert scale, focusing on perceived ease of use (PEU), perceived usefulness (PU), and behavioral intention (BI). The results revealed high mean scores, ranging from 3.75 to 4.58 for PEU, 4.08 to 4.25 for PU, and 4.25 to 4.50 for BI, with significant positive effects observed between PEU and PU, PEU and BI, and PU and BI. Qualitative feedback highlights preferences for interactive conversational user interfaces, the need for multimodal input capabilities, improvements in AI’s comprehension of BIM systems, and the importance of enhanced system stability and control over AI-generated results. Addressing these areas will improve adaptability, reliability, and user confidence, advancing LLM-driven BIM adoption in the AECO industry.