Model Evaluation and Selection

Model Evaluation and SelectionA comparison method was employed that combines the weights of the metrics mAP50, F1 Score, GPU Inference Time, and CPU Inference Time to calculate a score, S. The score S was calculated using the following formula:S = 0.3 * mAP50 + 0.3 * F1 + 0.1 * GPU Inference Time + 0.3 * CPU Inference TimeThe weights used were based on the importance of each metric in evaluating an object detection model. Inference time was considered a critical metric—as the goal is a fast and efficient model—assigning it a total weight of 0.5 (distributed across environments). Detection efficiency metrics, mAP50 and F1 Score, were considered equally important, with a weight of 0.25 each. This creates a balance between model efficiency and precision.The four models with the highest scores were chosen for the final comparison, distinguishing between GPU and CPU inference. The selected models were:Top 4 Models for GPU Index, YOLO Model, Image Size (px), Total Score (GPU) 11, 5su, 640, 0.960768 27, 9c, 640, 0.923060 19, 8n, 640, 0.914276 17, 8m, 640, 0.907099Top 4 Models for CPU Index, YOLO Model, Image Size (px), Total Score (CPU) 11, 5su, 640, 0.909831 19, 8n, 640, 0.887971 20, 8s, 320, 0.870448 6, 5mu, 320, 0.868559It is observed that the 5su model is the best for GPU inference, while the 5mu model is the best for CPU inference. Both models achieved a total score exceeding 0.9, indicating they are efficient and accurate models applicable to both inference environments.Optimization and Final SelectionFollowing the model selection, the YOLO models were converted to ONNX, OpenVINO, and TFLite formats. These formats were used to evaluate, compare, and select the optimal model for CPU and GPU inference. The models were evaluated based on mAP50, Precision, Recall, F1 Score, Speed (GPU, ms), and Speed (CPU, ms). The results across different formats were compared to finalize the best model for each hardware environment.