Convolutions Predictable Offloading to an Accelerator: Formalization and Optimization

Convolutional neural networks (CNNs) require a large number of multiply-accumulate (MAC) operations.To meet real-time constraints, they often need to be executed on specialized accelerators composed of an on-chip memory and a processing unit.However, the on-chip memory is often insufficient to store all the data required to compute a CNN layer.Thus, the computation must be performed in several offloading steps.We formalise such sequences of steps and apply our formalism to a state of the art decomposition of convolutions.In order to find optimal strategies in terms of duration, we encode the problem with a set of constraints.A Python-based simulator allows to analyse in-depth computed strategies.Accelerator DRAM MEM PE Input Kernel Kernel Output Store Output and Free unnecessary data (a) 1) free the memory and 2) store the results Accelerator DRAM MEM PE Input Kernel Kernel Input slice Loading / Kernel subset Loading (b) 3) load the elements Accelerator DRAM MEM PE Input Kernel Kernel Computation (c) 4) launch the computation