Descriptor: Synthetic Person Information Dataset for Entity Resolution (SPIDER)

Read full text
Authors: Praveen Chinnappa, Yash Mathur, ROSE MARY AROKIYA DASS
Cited by: 1
References: 11

Abstract

In the era of AI and cloud computing, customer data has become an integral part of modern business systems. As these data grow in volume and complexity, maintaining data quality, particularly managing duplicate records, remains a persistent challenge. Entity Resolution (ER) algorithms play a vital role in eliminating duplicates. However, developing, testing, and benchmarking these algorithms require large volumes of labeled data containing known duplicate records, which are rarely available in public datasets. The Synthetic Person Information Dataset for Entity Resolution (SPIDER) addresses this issue by providing 50,000 labeled person records, including 10,000 duplicates induced through seven rule-based transformations that simulate real-world duplicate variations. The dataset is available in JSON and CSV formats to ensure compatibility with standard software tools and includes a Python script for data generation to support reproducibility and extensibility. SPIDER offers a unified, person-level benchmark dataset with explicit duplicate lineage, advancing the capabilities of existing ER data-generation resources such as FEBRL and Magellan, and providing a reproducible foundation dataset for ER testing and evaluation.

Fields of Study & Subjects

Data Quality and Management Privacy-Preserving Technologies in Data Machine Learning in Healthcare

Publication details

Journal / source
IEEE data descriptions.
Publisher
Institute of Electrical and Electronics Engineers (IEEE)
Volume
3
Pages
121–131
Publication date
1 January 2026