Floating waste detection using deep learning: a comparative study of YOLO, RT-DETR, and faster R-CNN

Abstract Floating waste in inland water bodies poses severe threats to aquatic ecosystems, water quality, and public health. The accurate and timely detection of such waste is essential for enabling autonomous cleanup sys-tems like unmanned surface vehicles (USVs). However, detecting floating waste remains challenging due to the small size of debris, water surface reflections, glare, and complex backgrounds. This study presents a comparative evaluation of state-of-the-art deep learning-based object detection models—YOLO (v8–v10), Faster R-CNN, and Real-Time Detection Transformer (RT-DETR)—using the FloW-Img dataset, which is specifically designed for floating waste detection from USV perspectives. To enhance detection performance, we also explored four ensemble strategies: Weighted Box Fusion (WBF), Non-Maximum Suppression (NMS), Soft-NMS, and Non-Maximum Weighted (NMW). Our experiments show that the ensemble of RT-DETR-X and Faster R-CNN using WBF achieves the best results, with a mean Average Precision (mAP50) of 89.081%. This performance surpasses all previously reported methods on the same dataset, including YOLO-Float and Cascade R-CNN. The findings demonstrate the effectiveness of deep learning ensembles in improving small object detection in challenging water environments. This comparative study contributes valuable insights for developing robust, real-time, and scalable solutions for environmental monitoring and automated waste management systems.