AI Hardware & Software Stack

In post-training quantization of activations to INT8, what is the purpose of calibration?

  1. ARunning representative input data to observe each layer's activation range and set the quantization scales
  2. BPrecisely adjusting the GPU clock
  3. CAdjusting how colors appear on a screen
Show answer and explanation

Answer: A. Running representative input data to observe each layer's activation range and set the quantization scales

Weights are fixed once training ends, but activation values vary with the input. So a few hundred inputs resembling real usage are run through the model to observe each layer's activation range and distribution, and the range mapped onto INT8's -127 to 127 (the scale) is chosen. Matching the maximum lets outliers waste resolution, so the range is sometimes narrowed to cover most of the distribution.

  • A. Correct. How the range is chosen affects accuracy.
  • B. It is not clock adjustment.
  • C. It is not display adjustment.
Practice this topic Back to topic

More AI Hardware & Software Stack questions

  1. In quantizing large language models, a few channels with large outliers can hurt accuracy. What is a common countermeasure?
  2. In LLM inference, why does weight-only 4-bit quantization (W4A16, activations kept in FP16) especially speed up decode?
  3. Which statement about pruning neural networks is correct?
  4. Which describes the 2:4 structured sparsity supported by GPU matrix units?
  5. Which describes knowledge distillation?