Computer Arithmetic

What is the purpose of Kahan (compensated) summation when adding many floating-point numbers?

  1. ATo halve the number of additions and speed up the computation
  2. BTo convert all the floating-point numbers to integers before adding them
  3. CTo record lost parts and add them back, limiting error
Show answer and explanation

Answer: C. To record lost parts and add them back, limiting error

Each time a small value is added to a growing sum, its low-order digits are lost to absorption, and these losses accumulate into a large error. Kahan summation computes, after each addition, the difference between the actual increase and the value intended, records the lost part in a compensation variable, and adds it to the next value before summing. It costs several times more work but greatly limits error accumulation. It neither reduces the number of additions nor converts to integers.

  • A. The work actually increases.
  • B. It is not a method that converts to integers.
  • C. Correct. It records and restores lost parts, limiting error accumulation.
Practice this topic Back to topic

More Computer Arithmetic questions

  1. What is the advantage of computing a × b + c with an FMA (fused multiply-add) instruction?
  2. What is the main reason bfloat16 is used for AI training?
  3. When training neural networks in FP16 (half precision: 1 sign, 5 exponent, 10 significand bits), which problem tends to arise and is countered by scaling up the loss (loss scaling)?
  4. What is the largest finite value representable in FP16 (1 sign, 5 exponent bits with bias 15, 10 significand bits)?
  5. How are the two FP8 formats, E4M3 (4 exponent, 3 significand bits) and E5M2 (5 exponent, 2 significand bits), usually used?