Towards high-rank features on edge devices: a lightweight vision transformer with context-aware linear attention

Face recognition on resource-constrained edge devices demands models that are both computationally efficient and highly accurate. This paper presents a lightweight vision transformer that incorporates a novel Context-Aware Linear Attention (CALA) mechanism. This mechanism effectively overcomes the low-rank bottleneck in linear attention by dynamically recalibrating feature significance through global-context-derived adaptive coefficients applied to both the KV buffer and output space. As a result, it achieves near full-rank feature representations while maintaining linear computational complexity. The proposed architecture is rigorously optimized for edge deployment, incorporating streamlined linear projections and conditional positional encodings to minimize computational overhead. Additionally, we adopt the MagFace loss function, which introduces a quality-aware margin mechanism to improve discrimination, especially for low-quality images. Extensive experiments on benchmark datasets such as LFW, IJB-B, and IJB-C demonstrate that CALAFace achieves state-of-the-art accuracy—e.g., 99.72% on LFW and 95.3% on IJB-C—with only 2.3M parameters and 0.21 GFLOPs, outperforming existing lightweight models. The proposed model maintains optimal efficiency for edge deployment while significantly boosting robustness and recognition performance, offering a superior trade-off between accuracy and computational cost.