Low-Bit-Width Zero-Shot Quantization With Soft Feature-Infused Hints for IoT Systems

作者：Xinrui Chen, Yizhi Wang, Yao Li, Xitong Ling, Min Li, Ruikang Liu, Minxi Ouyang, Kang Zhao, Tian Gu, Yonghong He · 发表于：IEEE Internet of Things Journal · 年份：2024 · DOI：10.1109/jiot.2024.3507114 · 被引用次数：14 · 研究领域：Image Processing Techniques and Applications、CCD and CMOS Imaging Sensors、Advanced Image and Video Retrieval Techniques

Quantization has enabled the widespread implementation of deep learning algorithms on resource-constrained Internet of Things (IoT) devices, which compresses neural networks by reducing the bit-width of their parameters. However, most quantization methods invade privacy as they require real training datasets for calibration or fine-tuning. As a solution, zero-shot quantization (ZSQ) has emerged as a paradigm to quantize neural networks without accessing training datasets. Most employ data generation schemes to synthesize calibration data for knowledge transfer from the full-precision networks to the quantized ones. For privacy-protected and resource-constrained IoT devices, achieving optimal deployment necessitates the strategic integration of synthetic data generation and low-bit-width quantization techniques. However, when it comes to the lower bit-width case in ZSQ, we observe that the discrepancy between the full-precision network and the quantized network tends to widen significantly, hindering the knowledge transfer, which is attributed to the three following challenges: 1) hard logits matching with wide discrepancy; 2) unstable feature alignment with huge quantization error; and 3) synthetic data with low diversity. To address these issues, this article presents S-ZSQ, a novel ZSQ framework with two-pronged strategies that enhances both knowledge transfer and synthetic data generation, which enables low-bit-width quantized network to derive more soft feature-infused hi...