ASSESSMENT OF LARGE LANGUAGE MODELS (LLMS) EMPLOYED BY GENERATIVE AI

Read full text
Authors: Guanis de Barros Vilela, Ricardo Pablo Passos
Cited by: 0

Abstract

Introduction: Large Language Models (LLMs) are architectures employed in Genera-tive Artificial Intelligence (GAI) algorithms, such as those underlying ChatGPT, Gem-ini, Claude, Grok, among others. It is evident that an LLM can be highly efficient in performing a task X that requires substantial computational capacity, yet comparative-ly inefficient in executing a more specific task Y, which could be adequately handled by a Short Language Model (SLM), characterized by faster processing and significant-ly lower energy consumption. Beyond their high energy demands, GAI systems are in-herently fallible. Even when their performance metrics demonstrate excellent results in domains governed by strong consensus (tacit or consolidated knowledge)—which undoubtedly benefits millions of users worldwide—these systems may exhibit signifi-cant “hallucinations” when confronted with subjective content and/or borderline, am-biguous scenarios. The aim of this article was to evaluate the efficiency of hegemonic LLMs in responding to problem situations involving logical reasoning as well as is-sues characterized by high degrees of subjectivity. Methods: A set of ten open-ended questions was developed by researchers of recognized expertise, encompassing a wide range of topics—from logic and mathematics to subjective situations and challenges that are difficult to address at a global scale. These questions were submitted on the same day to the LLMs ChatGPT, Perplexity, Gemini, Claude, DeepSeek, and Grok, all in their most recent versions as of December 30, 2025. A typification matrix was de-veloped to categorize and analyze the response profiles generated by each model. Re-sults: Although current LLMs demonstrate satisfactory performance in retrieving in-formation grounded in scientific consensus— effectively addressing queries related to categorical and well-established scientific knowledge—they also engage with subjec-tive, philosophical, and humanitarian themes. In such contexts, however, a tendency toward imprecision and hallucination is observed. This phenomenon can be attributed to the statistical filtering intrinsic to these systems, which is designed to suppress “noise” but may inadvertently eliminate singularities and contextual nuances that are essential for the accurate understanding of non-standardized and highly contextualized scenarios.

Fields of Study & Subjects

Topic Modeling Artificial Intelligence Applications Ethics and Social Impacts of AI

Publication details

Journal / source
Revista CPAQV - Centro de Pesquisas Avançadas em Qualidade de Vida
Volume
18
Issue
1
Pages
28–28
Publication date
9 February 2026
Language
EN