A multimodal-input-enabled adaptive large language model question answering system
Summary
No summary yet. VIRAT can read this paper and write a plain-language summary of its aim, methods, key findings and limitations.
Summarise with VIRAT Free to try with your Science Catalogs account.Abstract
Large language models (LLMs) have radically transformed the model of human-computer interaction, yet the vast majority of question answering systems remain far too imited to handle multimodal input, give fexibility to their models, and offer strong reasoning for complex queries. To overcome these drawbacks, the design, implementation, and evaluation of the adaptive question answering system with multimodal input support are outlined herein in this dissertation. This study has developed a hybrid architecture of a system, combining a Next.js coordination layer and dedicated Python microservices. The architecture is capable of running a full multimodal content extraction ipeline that processes a wide variety of file types, including documents (e.g., PDF, DOCX, PPTX), images, audio, and video with the help of the latest models, including PaddleOCR, OpenAI Whisper, and BLIP. Functional analysis of this module has validated its effectiveness, achieving a Bag-of-Words F1-score of 0.832 on complex document extraction. We have developed a three-stage sophisticated hybrid intent recognition system that will intelligently direct user queries. This system is a hybrid of a finely tuned XLM-RoBERTa-large version as a high-speed domain expert and a GPT-4o Mini LLM version as a general reorderer. A classification accuracy of 93.88% was attained based on the dynamic weighted decision fusion logic with encoder confidence and entropy, which was far higher than that of the pure LLM baseline (approximately 75%) and the accuracy of the use of a fine-tuned encoder alone (89.69%). Furthermore, this study proposes and implements a novel collaboration paradigm, the “Thinker-Answerer.” By programmatically decoupling the reasoning (Chain-of-Thought generation) and synthesis (answer generation) stages, this paradigm automates user workflow processing. In qualitative case studies, it has been shown that unlike single-model techniques, the paradigm produces logically strong yet analytically rich solutions to complex queries. In summary, the dissertation offers a complete and tested prototype of one of the next-generation conversational artificial intelligence systems. It successfully integrates advanced multimodal processing technologies, high-precision intent recognition capabilities, and an innovative collaborative reasoning system.
Fields of Study & Subjects
Topic Modeling
Multimodal Machine Learning Applications
Natural Language Processing Techniques
Publication details
- Journal / source
- DR-NTU (Nanyang Technological University)
- Publication date
- 1 January 2026
- Language
- EN
- Document type
- Thesis