Descriptor: Synthetic Person Information Dataset for Entity Resolution (SPIDER)
Abstract
In the era of AI and cloud computing, customer data has become an integral part of modern business systems. As these data grow in volume and complexity, maintaining data quality, particularly managing duplicate records, remains a persistent challenge. Entity Resolution (ER) algorithms play a vital role in eliminating duplicates. However, developing, testing, and benchmarking these algorithms require large volumes of labeled data containing known duplicate records, which are rarely available in public datasets. The Synthetic Person Information Dataset for Entity Resolution (SPIDER) addresses this issue by providing 50,000 labeled person records, including 10,000 duplicates induced through seven rule-based transformations that simulate real-world duplicate variations. The dataset is available in JSON and CSV formats to ensure compatibility with standard software tools and includes a Python script for data generation to support reproducibility and extensibility. SPIDER offers a unified, person-level benchmark dataset with explicit duplicate lineage, advancing the capabilities of existing ER data-generation resources such as FEBRL and Magellan, and providing a reproducible foundation dataset for ER testing and evaluation.
Fields of Study & Subjects
Data Quality and Management
Privacy-Preserving Technologies in Data
Machine Learning in Healthcare
Publication details
- Journal / source
- IEEE data descriptions.
- Publisher
- Institute of Electrical and Electronics Engineers (IEEE)
- Volume
- 3
- Pages
- 121–131
- Publication date
- 1 January 2026