LLMs and Custom GPTs as a Tool for Academic Research

This study explores the accuracy and usefulness of OpenAI’s large language models (LLMs), particularly focusing on a customized GPT model trained on a specific dataset of musicology research. The research compares the performance of the custom model against the general ChatGPT-4.o model in an academic setting. The customized GPT, tailored to retrieve information from a controlled set of scholarly articles, was tested by a group of musicology experts, including authors whose works were part of the dataset. Results indicate that the custom GPT generally provides more precise and research-based responses compared to the general model. However, issues such as hallucinations, inconsistent performance, and ethical concerns regarding copyright and the use of LLMs in academic work were observed. The study concludes that while customized GPTs can enhance academic research, their use should be approached with caution due to ongoing challenges in accuracy and ethical implications.