D-Exit: A Decentralised Early Exit of Inference Framework for Swarm Systems

The increasing complexity and intensity of computational tasks related to Machine Learning necessitates flexible solutions that can be implemented across the IoT-edge-cloud continuum, relying on heterogeneous distributed deployments like swarm systems. These solutions are required to offer secure and reliable deployments, addressing key challenges such as the communication and computational resource scarcity and the near real-time needs of modern applications. This paper presents a decentralised early-exit of inference framework for swarm systems (D-Exit) based on a peer-to-peer communication, distributing the inference process across swarm nodes, reducing the computational burden of individual participants and minimizing the single point of failure risk. Numerical evaluations based on simulated swarms demonstrate the key metrics involved in the D-Exit framework, including the computational load of the nodes and the trade-off between the inference latency and the model accuracy.