TRUST Implementation based on the paper: Efficient Diverse Redundant DNNs for Autonomous Driving

This repository implements the TRUST algorithm presented in our work "Efficient Diverse Redundant DNNs for Autonomous Driving" at the 2023 IEEE 47th Annual Computers, Software, and Applications Conference (COMPSAC'23), on June 27-29, 2023. The paper can be accessed at: https://ieeexplore.ieee.org/document/10197016 This tool allows the execution of the TRUST algorithm which merges the detections from two object detection models. We refer the interested reader to the paper for specific details about the algorithm and setup. Note that we have tested the tool with the outputs from YOLOv4 in the Darknet framework (https://github.com/AlexeyAB/darknet). For example, running one model with FP16 and the other with FP13. The inference of each frame is averaged with the previous 2 frames, as implemented by the Darknet framework. The tool is designed for this specific 3-frame average. Tool parameters There are 3 parameters that must be passed to the TRUST program in order to execute it: DIFF_TH: threshold used for the TRUST algorithm N_VIDEOS: number of videos N_FRAMES: number of frames per video Directories needed mainmodel_detections/: folder that contains the detections from the main model submodel_detections/: folder that contains the detection from the secondary model TRUST_results_threshold_"X"/: folder to store the results of TRUST. Replace "X" by the DIFF_TH used (e.g., TRUST_results_threshold_0.100) (3 decimals) The files inside each folder must have the following format: video"X"_frame"X".txt (replace "X" by the actual video number and frame number) Format of input detections The detections in the directories mainmodel_detections/ and submodel_detections/ must have the following format for each detection made at frame_i: class_i avg_confidence_i avg_left_i avg_top_i avg_right_i avg_bot_i int_left_i int_top_i int_right_i int_bot_i int_conf_i int_object_probability_i int_class_probability_i frame_number_i class_i-1 avg_confidence_i-1 avg_left_i-1 avg_top_i-1 avg_right_i-1 avg_bot_i-1 int_left_i-1 int_top_i-1 int_right_i-1 int_bot_i-1 int_conf_i-1 int_object_probability_i-1 int_class_probability_i-1 frame_number_i-1 class_i-2 avg_confidence_i-2 avg_left_i-2 avg_top_i-2 avg_right_i-2 avg_bot_i-2 int_left_i-2 int_top_i-2 int_right_i-2 int_bot_i-2 int_conf_i-2 int_object_probability_i-2 int_class_probability_i-2 frame_number_i-2 where each line corresponds to the results of that specific detection at frames i, i-1, and i-2, respectively. This can be obtained from Darknet by printing the values of that specific output index for the frame_i-1 and frame_i-2 output arrays. For example: 2 0.502349 742 469 762 488 747 467 760 490 0.002956 0.764160 0.003868 5 2 0.748127 742 469 762 488 739 470 764 487 0.758282 0.759766 0.998047 4 2 0.330248 742 469 762 488 740 470 764 486 0.735631 0.737793 0.997070 3 Note that "avg_left_i-1 avg_top_i-1 avg_right_i-1 avg_bot_i-1" and "avg_left_i-2 avg_top_i-2 avg_right_i-2 avg_bot_i-2" can be left as the same values from frame_i, set to -1, or set to any other value, since only the average coordinates at frame_i are used in this algorithm. These values are as follows: class (int): class of the detection avg_confidence (float): average confidence at frame_i (averaged with frames i-1 and i-2) avg_left (int): average left coordinate at frame_i (averaged with frames i-1 and i-2) avg_top (int): average top coordinate at frame_i (averaged with frames i-1 and i-2) avg_right (int): average right coordinate at frame_i (averaged with frames i-1 and i-2) avg_bot (int): average bottom coordinate at frame_i (averaged with frames i-1 and i-2) int_left (int): internal left coordinate at frame_i int_top (int): internal top coordinate at frame_i int_right (int): internal right coordinate at frame_i int_bot (int): internal bottom coordinate at frame_i int_conf (float): internal confidence at frame_i int_object_probability (float): internal object probability at frame_i int_class_probability (float): internal class probability at frame_i frame_number (int): index of the frame where that detection was made Format of output detections The output detections have the following format: class avg_confidence avg_left avg_top avg_right avg_bot Example use Example execution with DIFF_TH=0.1, N_VIDEOS=1, N_FRAMES=5. We have provided zip files with 3 frames as an example, and the output that should be produced by the tool using the following command: .\TRUST.exe 0.1 1 5