Full-Network Post-Training Quantization for DETR with Integerized Nonlinear Operators and Scale Optimization
Summary
No summary yet. VIRAT can read this paper and write a plain-language summary of its aim, methods, key findings and limitations.
Summarise with VIRAT Free to try with your Science Catalogs account.Abstract
Quantization is an effective model compression technique that can significantly reduce storage footprint and inference cost. However, post-training quantization (PTQ) avoids additional training but often suffers from notable accuracy degradation. Existing PTQ methods mainly target computeintensive operators (e.g., convolutions and matrix multiplications) while inadequately handling nonlinear operators such as Layer Normalization (LN) and Softmax, which leads to frequent quantize/dequantize operations and mixed-precision fallbacks, introducing extra overhead and weakening end-to-end speedups. For DETR, a CNN-Transformer hybrid object detector, we propose a full-network PTQ approach: we reconstruct LN and Softmax into an integer inference chain to eliminate mixed-precision overhead, and further perform low-cost validation-subset-driven search to optimize weight/activation scales, reducing accumulated quantization errors without extra training. Experiments show that under full-network 8-bit quantization, our model achieves 36.3 mAP with only a minor drop from the full-precision baseline and outperforms existing methods.
Fields of Study & Subjects
Advanced Neural Network Applications
Domain Adaptation and Few-Shot Learning
Machine Learning and Data Classification
Publication details
- Journal / source
- International Conference on Electrical Automation and Artificial Intelligence (ICEAAI)
- Pages
- 1310–1314
- Publication date
- 16 January 2026
- Language
- EN