Ensemble Artificial Intelligence Framework for Real-Time Automatic Vehicle Detection and Classifications

Vehicle detection plays a vital role in a variety of applications such as autonomous driving, traffic management, and public safety. In this article, we propose an ensemble artificial intelligence framework to simultaneously detect and classify various vehicles on public roads. The framework aims to improve the detection accuracy of vehicles and their small targets at long distances in different environmental conditions: rainy, foggy, and night scenes. To achieve this goal, we first construct a benchmark vehicle dataset (Dataset 4) containing multiple vehicle types (Truck, Van, Car, Bus and Two-wheelers) across complex environmental conditions. Subsequently, we adopt multiple state-of-the-art AI detectors, including finetuned versions of YOLOv5, YOLOv7, YOLOv8, YOLOv10, DETR-ResNet50, and Faster R-CNN. Using the random-search approach the hyperparameters of the AI models are automatically optimized to select and fuse the top three models into a unified detection framework through an ensemble learning strategy. The ensemble framework aims to combine the advantages of multiple detectors to improve the detection accuracy and robustness under different vehicle types and conditions. Our proposed integrated model framework based on YOLOv7, YOLOv8 and YOLOv10 further improves the model performance using non-maximum suppression (NMS) and dynamic weighting methods, outperforming single detectors with excellent detection performance of 94.2% precision, 94% recall, 94.1% F1-score, and 96.6% mAP. The main contribution is the proposal of a novel vehicle detection integration framework and a comprehensive benchmark dataset to address the challenges faced by current vehicle detection systems. The system is deployed to test real driving videos, and the experimental results demonstrate the practical applicability of our proposed method.