What is INT8?
An 8-bit signed integer (−128 to 127). In AI inference, quantizing weights and activations to INT8 with a scale factor greatly cuts computation and memory and allows fast processing on small integer circuits.
Field: Computer Arithmetic
INT8 in the explanations
So a few hundred inputs resembling real usage are run through the model to observe each layer's activation range and distribution, and the range mapped onto INT8's -127 to 127 (the scale) is chosen.
In post-training quantization of activations to INT8, what …It is mostly used with low precisions such as FP16, BF16, FP8 and INT8, greatly speeding up the matrix products that dominate neural network training and inference.
What is a characteristic of the tensor cores (matrix units)…TOPS (trillions of operations per second) is the theoretical figure with every unit busy all the time, usually counted at INT8 or lower and sometimes assuming sparsity.
What should be kept in mind about AI accelerator performanc…
Questions that mention INT8 (5)
- In AI inference, what is the main benefit of converting (quantizing) weights from FP32 to INT8?
- Data ranging from −6.35 to 6.35 is quantized to symmetric INT8 (−127 to 127) centered at zero, with scale = maximum absolute value ÷ 127. Which integer represents 1.0?
- In post-training quantization of activations to INT8, what is the purpose of calibration?
- What is a characteristic of the tensor cores (matrix units) in recent GPUs?
- What should be kept in mind about AI accelerator performance figures quoted as "XX TOPS"?
Related terms
AI inference Post-training quantization Tensor core FP8 TOPS Arithmetic shift bfloat16 Booth's algorithm Carry-lookahead adder Carry-save adder