BackTranslationLLM: An Agentic AI Architecture for Translating and Validating the Content of Psychometric Instruments

Read full text
Authors: Frederico Gonçalves Pedrosa
Cited by: 0

Abstract

The cross-cultural adaptation of psychometric instruments is a methodologically complex, time-consuming, and resource-intensive process. This study introduces and evaluates BackTranslationLLM, a multi-agent Artificial Intelligence (AI) architecture designed to automate the translation and content validation of psychological tests. Guided by principles of AI agent engineering, the system employs a sequential pipeline with specialized agents for translation, back-translation, and refinement, followed by a diversified committee of AI judges for evaluation. The framework was applied to adapt the Cuestionario de Impacto de la Sesión de Musicoterapia from Spanish to Brazilian Portuguese. The results were benchmarked against a parallel validation conducted by five human expert judges. Exploratory Graph Analysis confirmed that the automated translation preserved the instrument's semantic structure. Both the AI and human committees assigned adequate Content Validity Coefficients (CVCs) to the adapted instrument. However, a lack of correlation between the item-level CVCs (e.g., Clarity, ρ = -0.37) revealed divergent judgment heuristics: the AI acted as a auditor of methodological compliance, whereas human experts excelled at evaluating pragmatic and cultural nuances. We conclude that BackTranslationLLM is an effective framework that complements, rather than replaces, human expertise by optimizing the validation process.

Fields of Study & Subjects

Health Education and Validation Psychometric Methodologies and Testing Natural Language Processing Techniques

Publication details

Publication date
17 January 2026
Document type
Preprint