An LLM-Native Psychometric Instrument: Factor-Analytic Derivation and Validation of Behavioral Disposition Scales for Large Language Models

Read full text
Authors: Juan Manuel Contreras
Cited by: 0

Abstract

This study develops and validates the first psychometric personality instrument whose constructs are derived from LLM behavioral affordances rather than human personality theory. Existing tools (MPI, TRAIT, DeepMind framework) apply human Big Five or Dark Triad inventories to LLMs; none define constructs native to LLM behavior such as sycophancy, refusal sensitivity, or hedging. We generate ~200 items across 13 candidate behavioral domains, administer them to 25 models spanning 16 families, and use exploratory factor analysis to derive a new latent trait structure bottom-up. The instrument undergoes a full construct-validity pipeline: internal consistency, cross-run stability, convergent/discriminant validity against BFI-44, and predictive validity via blinded human ratings of independent behavioral samples. All materials, data, and code will be released as open-source.

Fields of Study & Subjects

Personality Traits and Psychology Ethics and Social Impacts of AI Psychometric Methodologies and Testing

Publication details

Publication date
1 January 2026
Language
EN
Document type
Preprint