Computer Arithmetic

Which gives the approximate number of reliable decimal digits for single and double precision?

  1. AAbout 3 for single, about 7 for double
  2. BAbout 7 for single, about 15–16 for double
  3. CAbout 32 for single, about 64 for double
Show answer and explanation

Answer: B. About 7 for single, about 15–16 for double

Significant digits are set by significand precision. Single precision has 24 bits including the implied 1, and 2²⁴ ≈ 1.7 × 10⁷, so about seven decimal digits; double has 53 bits, and 2⁵³ ≈ 9 × 10¹⁵, so about 15–16 digits. Storing 123456789 in single precision, for example, yields 123456792, losing the low digits. 32 or 64 digits mistakes the total bit count for decimal digits, and about three digits is closer to half precision.

  • A. About three digits is closer to half precision.
  • B. Correct. About seven for single and 15–16 for double.
  • C. This mistakes bit counts for decimal digits.
Practice this topic Back to topic

More Computer Arithmetic questions

  1. What is the role of IEEE 754 subnormal numbers?
  2. On some processors, computations involving many subnormal numbers can slow down dramatically. Which setting is used when performance takes priority?
  3. Under IEEE 754, what result does the standard specify for dividing a positive number by zero?
  4. When x is NaN, what does evaluating x == x give in most programming languages?
  5. In IEEE 754 single precision, what do bit patterns with an all-ones exponent field (255) represent?