Data-Free Post-Training Quantization Method and Apparatus, Device, and Storage Medium

Smart overview of the Invention

The patent application introduces a novel method for data-free post-training quantization, which allows for the conversion of floating-point model parameters to fixed-point parameters without needing standard datasets. This process reduces the computational demands typically associated with training a data generation model. By leveraging user-provided data distributions and random data inputs, the method facilitates efficient quantization of neural network models, enhancing inference performance while conserving resources.

Technical Approach

Key to the method is obtaining a user-defined data distribution that represents the expected input activation values for each network layer. Random data is processed through the floating-point model to generate these activation values, which are then statistically analyzed to define their data range. Quantization parameters are derived from these ranges, enabling the quantization of activation values during model inference. This approach bypasses the need for a standard dataset, significantly reducing resource consumption.

Optimization Techniques

An optional implementation involves iteratively refining the quantization parameters to minimize quantization errors. This is achieved by adjusting initial quantization parameters based on endpoint values and using the quantization error as a constraint. Additionally, when dealing with scale factors and quantization offsets, an alternating update method is employed to optimize these parameters further, ensuring minimal quantization error.

Weight Value Quantization

The method can also extend to quantizing weight values within the floating-point model. By grouping weights and determining corresponding quantization parameters, the method reduces storage and computational resource demands. For weights with significant quantization errors, the rounding direction is adjusted using disturbance tensors, further refining the quantization process. This adjustment process is tailored to both symmetric and asymmetric quantization scenarios.

Apparatus and Implementation

The patent also describes a data-free post-training quantization apparatus capable of implementing the method. This apparatus can be realized through software, hardware, or firmware, and includes modules designed to execute the specified quantization process. Furthermore, a compute device and a computer-readable storage medium are proposed, both containing instructions to carry out the described method, ensuring comprehensive application across various computational environments.