English
Menu

What is Rounding error?

The error that arises when a real number is represented with a finite number of digits and a value that cannot be represented is rounded to a nearby one. 0.1 cannot be represented exactly as a binary floating-point number, so errors build up when it is added repeatedly, and adding 0.1 ten times in double precision does not give 1.0. Avoid comparing real numbers with = and counting loop iterations with real numbers.

Japanese name: 丸め誤差

Rounding error appears in 5 practice questions across 2 topics, including Computer Arithmetic, Programming Basics (Pseudocode Tracing).

Field: Programming Basics (Pseudocode Tracing)

Try it: questions about Rounding error

What is the effect of not storing the leading 1 of the significand (the hidden bit) in IEEE 754 normalized numbers?

  1. AOmitting the leading 1 makes negative numbers representable
  2. BPrecision increases by one bit for the same number of stored bits
  3. COmitting the leading 1 makes results always exact
Show answer and explanation

Answer: B. Precision increases by one bit for the same number of stored bits

Normalized in binary, every nonzero number has the form 1.xxx × 2^exponent, so the leading digit is always 1. There is no need to store a known 1, so it is omitted and the bit is used for the fraction instead. Single precision thus stores 23 significand bits but has 24 bits of precision. Subnormals, with an all-zero exponent, treat the leading digit as 0. Negative numbers use the sign bit, and rounding error does not disappear.

Rounding to integers with round half to even, what do 2.5 and 3.5 become?

  1. A3 and 4
  2. B2 and 4
  3. C2 and 3
Show answer and explanation

Answer: B. 2 and 4

Both are exact ties, so each rounds to the even neighbor: 2.5, halfway between 2 and 3, goes to 2; 3.5, halfway between 3 and 4, goes to 4. Always rounding ties up would give 3 and 4, summing to 7, one more than the original sum of 6, whereas rounding to even gives 2 + 4 = 6, preserving the sum. 2 and 3 result from always rounding ties down. Avoiding biased rounding error is the method's strength.

Rounding error in the explanations

Questions that mention Rounding error (5)

  1. What is the effect of not storing the leading 1 of the significand (the hidden bit) in IEEE 754 normalized numbers?
  2. Rounding to integers with round half to even, what do 2.5 and 3.5 become?
  3. What is the phenomenon called in which subtracting two nearly equal floating-point numbers greatly reduces the significant digits?
  4. Why is fixed-point rather than floating-point arithmetic often used in small microcontrollers and DSPs?
  5. Real values are represented in IEEE 754 double precision (binary64), and each sum is rounded to the nearest representable value. What is the value of c after the following program runs?

Books on Programming Basics (Pseudocode Tracing) Ad

As an Amazon Associate I earn from qualifying purchases.

Related terms

IEEE 754 Significand Subnormal number Round half to even Catastrophic cancellation Fixed point Variable Argument Array Bitwise operation

Back to the glossary