Full-Network Post-Training Quantization for DETR with Integerized Nonlinear Operators and Scale Optimization

View at publisher Summarise with VIRAT
Authors: Yuhao Shao, RongChun Li, Peng Qiao, Jingfei Jiang
Cited by: 0
References: 21

Summary

No summary yet. VIRAT can read this paper and write a plain-language summary of its aim, methods, key findings and limitations.

Summarise with VIRAT Free to try with your Science Catalogs account.

Abstract

Quantization is an effective model compression technique that can significantly reduce storage footprint and inference cost. However, post-training quantization (PTQ) avoids additional training but often suffers from notable accuracy degradation. Existing PTQ methods mainly target computeintensive operators (e.g., convolutions and matrix multiplications) while inadequately handling nonlinear operators such as Layer Normalization (LN) and Softmax, which leads to frequent quantize/dequantize operations and mixed-precision fallbacks, introducing extra overhead and weakening end-to-end speedups. For DETR, a CNN-Transformer hybrid object detector, we propose a full-network PTQ approach: we reconstruct LN and Softmax into an integer inference chain to eliminate mixed-precision overhead, and further perform low-cost validation-subset-driven search to optimize weight/activation scales, reducing accumulated quantization errors without extra training. Experiments show that under full-network 8-bit quantization, our model achieves 36.3 mAP with only a minor drop from the full-precision baseline and outperforms existing methods.

Fields of Study & Subjects

Advanced Neural Network Applications Domain Adaptation and Few-Shot Learning Machine Learning and Data Classification

Publication details

Journal / source
International Conference on Electrical Automation and Artificial Intelligence (ICEAAI)
Pages
1310–1314
Publication date
16 January 2026
Language
EN