Challenges in Managing Noisy Labels in Crowdsourced Construction Image Datasets

Accurate labeling of construction image datasets is vital for developing reliable machine-learning models for structural health monitoring and maintenance scheduling. Crowdsourcing offers a cost-effective labeling approach but introduces noise due to subjective judgments and varying annotator expertise. This study evaluates three label integration methods—majority voting (MV), weighted majority voting (WMV), and iterative weighted majority voting (IWMV)—using a synthetically labeled SDNET 2018 bridge deck crack detection dataset. We compare the impact of integrated label quality on model performance by training four convolutional neural networks with ground truth, MV, WMV, and IWMV labels. Although IWMV yields higher label accuracy than MV and WMV, all models trained on integrated labels still significantly underperform compared to the model using ground truth. These results demonstrate the limitations of current fusion techniques for noisy construction imagery and emphasize the necessity for novel, tailored integration strategies to address the unique challenges of construction dataset labeling.