LASER-NASA-TLX: Sprachmodell-gestützte Suchstrategien zum NASA Task Load Index in der Medizin (LASER = LLM-Assisted Search and Evidence Review)
Summary
No summary yet. VIRAT can read this paper and write a plain-language summary of its aim, methods, key findings and limitations.
Summarise with VIRAT Free to try with your Science Catalogs account.Abstract
The aim of the research project is to investigate the performance of large language models (LLMs) in generating systematic search strategies in PubMed on the topic of NASA-TLX. Five LLMs are used to create search strategies for systematic literature research on the topic of NASA-TLX in PubMed. The following models are used for this purpose: ChatGPT (version 5, OpenAI) , Gemini (Gemini 2.5 Flash), Grok (created by xAI), Mistral (LeChat, Mistral AI model) and Claude (Anthropic). A zero-shot prompt is used for the work. The work instructions for the language models (LLMs) deliberately avoid the use of technical terms in order to test the ability of language models to create a functional and sophisticated search structure from a colloquially formulated query. Therefore, no reference is made to the use of Medical Subject Headings (MeSH) and Boolean operators; however, models are free to use any appropriate PubMed syntax. The search strategies created by the language models are then tested for functionality using the search function of the PubMed platform. The primary outcome of this study is the F1 score of each LLM-based search strategy compared to the gold standard reference set. Secondary outcomes include sensitivity (recall), specificity, precision, and absolute numbers of retrieved and missed relevant records. The quality of the search strategies (e.g., correct syntax, MeSH use, variants of NASA-TLX) is defined as an exploratory outcome. Inter-rater reliability (e.g., Cohen’s kappa) will be assessed descriptively and reported separately from the primary outcomes. As part of the analysis of the LLMs, it is expected that performance differences between the models will become apparent in terms of the quality and completeness of the generated search strategies. In addition, varying hit counts for the respective search strategies in the databases are to be expected. The results of the work are intended to stimulate discussion on the extent to which search strategies generated by LLMs can support the creation of systematic reviews in the future.
Fields of Study & Subjects
Information Retrieval and Search Behavior
Artificial Intelligence in Healthcare and Education
Meta-analysis and systematic reviews
Publication details
- Publication date
- 1 January 2026
- Language
- EN
- Document type
- Preprint