A Disciplined Approach to Neural Network Framework Selection for On-board Critical Applications

Transforming Artificial Neural Networks (ANNs) into efficient executables on resource-constrained embedded platforms is an essential step for modern AI applications. This process relies on deployment toolchains, whose growing number and features raise a significant challenge for developers. Differences among these toolchains can have critical impacts on final system performance and development cost. To address this, our work introduces a disciplined approach comprising key evaluation criteria for systematically assessing and comparing neural network deployment frameworks. We illustrate our method through an extensive comparative analysis of leading toolchains targeting diverse hardware architectures, including FPGAs, GPUs, and CPUs / MCUs. The insights and practical guidelines derived from this study are intended to facilitate navigation in the complex toolchain landscape and help to take rational design and implementation decisions.