Are All Marine Species Created Equal? Performance Disparities in Underwater Object Detection

Underwater object detection remains challenging as degraded imagery, class imbalance, and species diversity all contribute to performance variations that are not yet well understood. We therefore ask two questions: 1) What factors beyond data quantity drive class-specific performance disparities? 2) How can we systematically improve detection of underperforming marine species? Using manipulated DUO and RUOD datasets, we decouple localization from classification and investigate the scallop class as a representative case study. Localization analysis using YOLO11 and TIDE shows that foreground-background discrimination is the most problematic stage regardless of how much training data is available. Classification experiments reveal persistent precision gaps even under balanced sampling, indicating intrinsic feature-based challenges beyond data scarcity. We recommend imbalanced distributions when prioritizing precision, and balanced distributions when prioritizing recall. Addressing under-performing classes in future work will require algorithmic advances, especially within localization modules. We publicly release our code and datasets.