Statuario-40k: Fine-tuning SAM 3 for instance segmentation at scale of monuments, statues, and cracks

Among the diverse pathologies that compromise the structural integrity of historical monuments, statues, and masonry infrastructures, surface cracking is a primary indicator of both localised material fatigue and systemic structural distress. However, instance segmentation of cultural heritage artefacts remains bottlenecked by annotation scarcity, strong geometric variability in sculptural forms, heterogeneous materials, scales, and preservation states, and the limited cultural breadth and variety of existing benchmarks. Moreover, instance segmentation of cracked statues and monuments is especially challenging because the fine-grained and unpredictable geometry of structural damage further complicates reliable annotation. In this paper we present Statuario-40k , a 39,280-image dataset and training pipeline for instance segmentation of monuments, statues, columns, and pedestals in challenging heritage imagery. Our key idea is to turn publicly available clean monument videos into a scalable pseudo-labelling data flywheel using the standard public SAM 3 video model with temporal disambiguation and to complement fixed prompt banks with a second, VLM-guided prompt discovery stage that mines video-specific three-word prompts. Starting from 247,286 video frames extracted from YouTube, we retain 37,643 video-derived images after prompt pruning, COCO-format merging, and instance de-duplication; we then add 1537 curated web images from Bing and Kaggle to obtain the final dataset. We further collapse heterogeneous prompt strings into four canonical heritage classes and fine-tune an image-only SAM 3 model by freezing the visual backbone while updating the segmentation head and only four layers of the language encoder, reducing GPU memory requirements. Finally, we propose a depth-aware crack synthesis pipeline that injects material-compatible defects from OmniCrack30k into clean heritage imagery by exploiting both the monocular depth estimates from Depth Anything and the semantic information already available in the dataset. This synthesis stage converts clean heritage images into crack-aware training data with depth- and semantics-consistent defect placement, enabling the training of models that detect and segment crack instances on cultural heritage structures. We provide the full methodological pipeline and implementation and training details, as well as quantitative and qualitative evaluation protocols for both heritage object and crack instance segmentation. The dataset is available for download at this link: http://deeplearning.ge.imati.cnr.it/statuario-40k .