Computer Arithmetic

What is the phenomenon called in which subtracting two nearly equal floating-point numbers greatly reduces the significant digits?

  1. AAbsorption
  2. BOverflow
  3. CCatastrophic cancellation
Show answer and explanation

Answer: C. Catastrophic cancellation

With seven significant digits, for example, 1.234567 − 1.234566 = 0.000001: the top six digits cancel, leaving only one significant digit. Tiny rounding errors in the original numbers become relatively huge in the result. This is catastrophic cancellation, countered by rearranging formulas to avoid subtracting close values. Absorption is a different phenomenon in which the smaller operand's digits are lost when adding numbers of very different magnitude.

  • A. Absorption is a different phenomenon in additions of unequal magnitudes.
  • B. Overflow is exceeding the representable range.
  • C. Correct. Subtracting close numbers loses significant digits.
Practice this topic Back to topic

More Computer Arithmetic questions

  1. For very large x, computing √(x + 1) − √x in floating point suffers catastrophic cancellation. Which rewrite computes it accurately?
  2. In single precision, adding 1 to 100 million (10⁸) leaves the result unchanged at 100 million. Which explains this?
  3. What is the purpose of Kahan (compensated) summation when adding many floating-point numbers?
  4. What is the advantage of computing a × b + c with an FMA (fused multiply-add) instruction?
  5. What is the main reason bfloat16 is used for AI training?