AI at the Edge - how tiny it can get

Many edge-computing applications require regression models to run directly on microcontroller-class hardware under tight memory, latency, and energy budgets. On such platforms, floating-point inference is often too costly or not available at all, which motivates the use of compact integer-only neural networks. However, for continuous outputs, quantization can severely degrade accuracy, and the combined effect of different compression techniques on regression performance remains insufficiently understood.This thesis investigates how effectively the performance of a small fully connected neural network for regression can be preserved under aggressive compression, with the goal of deployment on microcontroller-class devices. To this end, multiple compression and optimization strategies are systematically investigated. A synthetic but non-trivial benchmark is constructed based on the scalar mapping fθ(x, A) ≈ A cos(x), combining periodic structure with amplitude modulation. A manually implemented neural network framework is developed in Python, supporting integer-only inference, post-training quantization (PTQ), quantization-aware training (QAT), knowledge distillation (KD), and a custom integer-level simulated annealing (SA) procedure for post-training refinement.Within this framework, a wide floating-point teacher network and a compact student architecture are trained and combined into seven configurations (A)–(G) that systematically explore PTQ, KD, QAT, and SA. The results demonstrate that, in this controlled setting, combining KD, QAT, and discrete post-training refinement can substantially mitigate the accuracy loss induced by aggressive 8-bit quantization, while yielding models that are small enough for deployment on microcontroller-class edge devices.