Data source effects in research performance assessment of individuals and institutions: comparing OpenAlex with established bibliographic databases

Abstract Performance metrics inform hiring, promotion, and funding decisions; however, the choice of bibliographic database can substantially influence these outcomes. OpenAlex has emerged as a significant open alternative to proprietary sources; however, its suitability for research evaluation remains uncertain. Existing comparative studies across databases have largely focused on coverage and product-level assessments. In contrast, our work directly compares performance scores and rankings of scholars and institutions across data sources to identify the origins of any discrepancies. We rank the productivity and impact of Italian universities and professors in management engineering using OpenAlex, Web of Science (WoS), and Scopus. Productivity is measured using harmonized document types and author name disambiguation, while impact is derived from year- and field-normalized citations weighted by fractional authorship. We find strong agreement across databases for productivity, but only moderate alignment for impact. OpenAlex displays more heavily right-skewed distributions and greater dispersion, producing higher volatility in scholar and institutional rankings. Median absolute percentile differences are approximately 5–6 points across database pairs, with extremes exceeding 70 points for impact; around 30% of scholars shift at least one quartile when OpenAlex replaces WoS or Scopus. These discrepancies stem primarily from differences in field taxonomies, normalization procedures, and indexing-year conventions. We conclude that although OpenAlex enables open and reproducible analyses, high-stakes evaluations should triangulate multiple data sources and harmonize methodological assumptions in advance to ensure validity and reliability.