Computer Arithmetic

Data ranging from −6.35 to 6.35 is quantized to symmetric INT8 (−127 to 127) centered at zero, with scale = maximum absolute value ÷ 127. Which integer represents 1.0?

  1. A1
  2. B127
  3. C20
Show answer and explanation

Answer: C. 20

The scale is 6.35 ÷ 127 = 0.05, so integer 1 corresponds to the real value 0.05. A real value is quantized by dividing by the scale and rounding to the nearest integer: 1.0 ÷ 0.05 = 20. Dequantizing gives 20 × 0.05 = 1.0. 1 wrongly takes the real value as the integer, and 127 corresponds to the maximum, 6.35. Large outliers make the scale coarse and hurt accuracy, so choosing the range (calibration) matters.

  • A. 1 wrongly uses the real value directly as the integer.
  • B. 127 corresponds to the maximum value 6.35.
  • C. Correct. Dividing by the scale 0.05 gives 20.
Practice this topic Back to topic

More Computer Arithmetic questions

  1. Why is fixed-point rather than floating-point arithmetic often used in small microcontrollers and DSPs?
  2. Which gives the resolution and range of the 16-bit signed fixed-point Q15 format (15 fraction bits)?
  3. When two fixed-point numbers with 8 fraction bits are multiplied using an integer multiplier, how should the result be handled?
  4. Which describes saturating arithmetic used in signal processing?
  5. In a carry-lookahead adder, which gives the definitions of each bit's generate signal g and propagate signal p (a and b are that bit's inputs) and the carry into the next bit, c(i+1)?