From Keywords to Semantics: Perceptions of Large Language Models in Data Discovery

Despite attempts to increase metadate quality, researchers still find data discover challenging. One difficulty is the need to select appropriate keywords by knowing the research question and disciplinary terminology before searching. Large Language Models (LLMs) could make this process easier by parsing natural language nuances in metadata and user queries. While advances continue to be made with LLMs, little is known about user needs for adapting them to data discovery. In contrast, our work uses a human-centered artificial intelligence framework to run focus groups (N = 33) and better understand users’ perspectives towards LLMs for data discovery. Although users view LLMs as a promising technology, concerns about bias, hallucinations, and ethics hinder full acceptance. Unlike prior work, we operationalize transparency for information retrieval by identifying user-informed features that can help overcome these barriers. Our findings demonstrate why technological advances alone are not enough for users to trust and accept LLMs.