BIG-Bench Hard (BBH) Data Quality Investigation Report

Zenodo (CERN European Organization for Nuclear Research) 2026 Open access

Read full text
Authors: Yeosun Kyung
DOI: 10.5281/zenodo.18123658
Cited by: 0

Abstract

This report documents three ground truth label errors discovered in the BIG-Bench Hard (BBH) benchmark dataset. The errors affect date_understanding and geometric_shapes tasks, causing correctly-reasoning models to be marked as incorrect. Mathematical proofs and reproducible verification scripts are provided.

Publication details

Journal / source
Zenodo (CERN European Organization for Nuclear Research)
Publication date
2 January 2026