Uncertainty-Aware Mixture Models of First Significant Digit Frequencies Detect Irregularities in Scientific Data

Big data is the essence of modern research. Therefore, data integrity matters more than ever, motivating scalable tools for detecting anomalies across large collections of datasets. A promising yet underexplored approach is Benford's law, which posits that in naturally occurring datasets across several domains, smaller digits occur more frequently as leading digits. Deviations from this pattern can serve as an anomaly signal, flagging datasets for targeted expert review. However, quantifying the degree of adherence to Benford's law and estimating the associated uncertainty is challenging, especially across datasets of varying sizes. We introduce Simulation-Based Benfordness Estimation (SBBE), a Bayesian framework that models an observed dataset as a mixture of a Benford-conforming ("natural") component and a non-conforming component. SBBE returns uncertainty-aware estimates and enables interpretability and comparisons across datasets of different sample sizes. We validate SBBE on simulations and real-world datasets known to follow Benford's law. Finally, we apply SBBE to a bioactivity data case study, where rich metadata enable targeted follow-up analyses. Using SBBE, we detect irregularities in a highly curated database and trace likely sources of deviation, demonstrating that SBBE can serve as a domain-agnostic proxy for data quality.