Low-cost real-time inference for semantic segmentation of optical coherence tomography data

The study demonstrates real-Time semantic segmentation of optical coherence tomography (OCT) data using an edge AI processor (HAILO-8) for accelerated and power-efficient data analysis. Multiple Short-Term Dense Concatenate (STDC) networks, trained with publicly available middle ear and retinal OCT datasets, were optimized and quantized via the Hailo Dataflow Compiler. By converting 3D volumes of middle ear scans into 2D slices for sequential segmentation, i.e. similar to real-Time segmentation during data acquisition, inference speeds of 176fps in standard mode and 290fps in performance mode were achieved, approaching the demands of a 200kHz OCT systems while consuming minimal power. Accuracy improved when using three consecutive B-scan slices, though 2D inference slightly lagged behind GPU-based 3D approaches. In case of the retinal OCT dataset and a layer segmentation as required for several diagnostic tasks, inference speeds of 365 fps in standard mode and 627 fps in performance mode were achieved. Therefore, edge accelerators are a promising component for cost-and power-efficient data processing pipelines in real-Time OCT applications.