An LLM-Native Psychometric Instrument: Factor-Analytic Derivation and Validation of Behavioral Disposition Scales for Large Language Models
Abstract
This study develops and validates the first psychometric personality instrument whose constructs are derived from LLM behavioral affordances rather than human personality theory. Existing tools (MPI, TRAIT, DeepMind framework) apply human Big Five or Dark Triad inventories to LLMs; none define constructs native to LLM behavior such as sycophancy, refusal sensitivity, or hedging. We generate ~200 items across 13 candidate behavioral domains, administer them to 25 models spanning 16 families, and use exploratory factor analysis to derive a new latent trait structure bottom-up. The instrument undergoes a full construct-validity pipeline: internal consistency, cross-run stability, convergent/discriminant validity against BFI-44, and predictive validity via blinded human ratings of independent behavioral samples. All materials, data, and code will be released as open-source.
Fields of Study & Subjects
Personality Traits and Psychology
Ethics and Social Impacts of AI
Psychometric Methodologies and Testing
Publication details
- Publication date
- 1 January 2026
- Language
- EN
- Document type
- Preprint