SATNet: Low Complexity Encoder-Only Edge Transformer Design Using Second-Order Approximation for Road Classification

While Transformers dominate machine learning with state-of-the-art accuracy, their deployment on resource-constrained edge platforms is severely restricted by substantial computational complexity and power demands. Specifically, the hardware realisation of Transformers faces limitations due to the resource-intensive computations associated with Batch Normalisation (BN) and Gated Linear Unit (GLU) modules, which are identified as the most computationally demanding components within the architecture. To address this challenge and enable efficient deployment on resource-limited devices, we propose a second-order approximation encoder-only Transformer Network (coined as SATNet) architecture. SATNet firstly optimises theBNmodule, reducing its computational overhead; and secondly, apiece-wise second-order approximationfor the GLU activation function to minimise latency and energy consumption. Validated on diverse edge devices for intelligent vehicular road classification (utilising time-series sensor speed data from the wheels of a vehicle), proposed SATNet delivers transformative efficiency gains. A 2X–6.9X reduction in energy, a 1.7X–7.49X drop in latency, and a 15.2% reduction in memory usage. These results demonstrate that high-performance edge AI requires not just smaller models, but smarter, mathematically optimised architectures.