PSYCH—Psychometric Assessment of Large Language Model Characters: An Exploration of the German Language
Abstract
Background: Existing evaluations of large language models (LLMs) largely emphasize linguistic and factual performance, while their psychometric characteristics and behavioral biases remain insufficiently examined, particularly beyond English-language contexts. This study presents a systematic psychometric screening of LLMs in German using the validated Big Five Inventory-2 (BFI-2). Methods: Thirty-two contemporary commercial and open-source LLMs completed all 60 BFI-2 items 60 times each (once with and once without having to justify their answers), yielding over 330,000 responses. Models answered independently, under male and female impersonation, and with and without required justifications. Responses were compared to German human reference data using Welch’s t-tests (p
Fields of Study & Subjects
Psychometric Methodologies and Testing
Mental Health via Writing
Neurobiology of Language and Bilingualism
Publication details
- Journal / source
- Analytics
- Publisher
- MDPI AG
- Volume
- 5
- Issue
- 1
- Pages
- 5–5
- Publication date
- 6 January 2026
- Language
- EN